Motion area neural signal decoding method and device and storage medium
By processing EEG signals through a cascaded sparse autoencoder structure, the problem of unstable decoding performance of SVM and SRM on large data sets is solved, and stable and efficient decoding of motor area neural signals is achieved.
Patent Information
- Application Number
- CN202510792098.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-13
- Publication Date
- 2025-09-09
AI Technical Summary
In the existing technology, the support vector machine (SVM) decoding method performs poorly on large datasets, while the sparse representation method (SRM) that extracts sparse spatial features in neural responses ignores the temporal dynamics and correlations in neural responses, resulting in unstable decoding performance.
A cascaded sparse autoencoder structure was adopted. The EEG signals of multiple subjects were preprocessed and divided into 8 categories according to the direction of visual motion stimulation. The average response signal matrix was obtained, and the spatiotemporal features were extracted through the cascaded sparse autoencoder model. Iterative training was performed using a preset loss function to obtain a trained model to decode the direction of motion stimulation.
It achieves stable decoding performance on large data sets, has the advantages of robustness, low storage and low computational burden, can effectively extract the spatiotemporal features in the neural signals of the motor area, and improves the decoding accuracy.
Smart Images
Figure CN120611259A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of motor area neural signal decoding, and in particular to a motor area neural signal decoding method, device and storage medium. Background Art
[0002] Neural coding and decoding aims to use a variety of brain signals, such as action potentials, local field potentials, EEG signals and functional magnetic resonance imaging signals, to infer the behavior, perception and cognitive state of the subjects. It has important applications in neuroscience research, control and rehabilitation engineering.
[0003] Because neural population decoding can provide more accurate perception and behavioral predictions by leveraging the response information carried by multiple neurons rather than a single neuron, many such decoding methods are widely used in neuroscience. Support vector machines (SVMs) are one standard population decoding method, which typically compress population neural signal information by performing temporal averaging of neural responses. SVM decoding methods, which perform well on small and medium-sized datasets by finding the optimal classification hyperplane between different types of neural signal responses, perform poorly on large datasets because solving the support vectors requires a large amount of memory and long training times. Sparse representation methods (SRMs), another commonly used encoding and decoding method for population neural signal responses, extract sparse spatial features from neural responses. Although they have low storage and computational burdens, their decoding performance is unstable because they ignore the temporal dynamics and correlations in neural responses. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide a method, device and storage medium for decoding motor area neural signals to solve the problems in the prior art that the SVM decoding method performs poorly on large data sets, and the sparse representation method (SRM) for extracting sparse spatial features in neural responses ignores the temporal dynamics and correlations in the neural responses, and its decoding performance is unstable.
[0005] According to a first aspect of an embodiment of the present invention, a method for decoding motor area neural signals is provided, the method comprising: Collecting EEG signals from multiple subjects and performing signal preprocessing on the collected EEG signals; The preprocessed EEG signals were divided into 8 categories according to the direction of visual motion stimulation, and the average response signal of each category was obtained; Obtain the average response data matrix of each category based on the average response signal of each category; concatenate the eight average response data matrices by column to obtain the input data matrix; Inputting the input data matrix into a pre-built cascaded sparse autoencoder structure, extracting the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, and iteratively training the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model; The EEG signal to be decoded is converted into an input data matrix and input into the trained cascaded sparse autoencoder model. The trained cascaded sparse autoencoder model is used to extract the spatiotemporal features of the input EEG signal, and the prediction result of the motion stimulation direction is output based on the extracted spatiotemporal features.
[0006] Preferably, The collecting of EEG signals of multiple subjects includes: Having multiple subjects wear an EEG cap containing multiple electrodes, where the position and number of electrodes of the EEG cap meet preset standards; A plurality of subjects were asked to accept a visual motion stimulation task, and EEG signals were continuously collected at a certain sampling frequency. The visual motion stimulation task included observer displacement stimulation in front, back, left, and right directions in a virtual space.
[0007] Preferably, The signal preprocessing of the collected EEG signal includes: The collected EEG signal is filtered using a bandpass filter to remove high-frequency noise and low-frequency drift, wherein the bandpass range of the bandpass filter is 0.5-100 Hz; The independent component analysis method is used to remove artifact interference from the filtered EEG signal; The EEG signals after artifact interference removal were z-normalized to make the EEG signals of different subjects or different time periods comparable.
[0008] Preferably, The pre-built cascaded sparse autoencoder structure consists of three parts: a cascaded time-domain sparse autoencoder, a frequency-domain sparse autoencoder, and a classifier; The input layer of the time-domain sparse autoencoder is provided with X artificial neurons, the output layer is provided with X artificial neurons, and the number of neurons in the hidden layer is 1; The number of nodes in the input layer and output layer of the frequency domain sparse autoencoder is set to N , which is equal to the number of EEG signal channels in the motor area, sets the number of hidden layer nodes after debugging; The classifier is a softmax classifier.
[0009] Preferably, The step of inputting the input data matrix into a pre-built cascaded sparse autoencoder structure, extracting the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, and iteratively training the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model, comprising: Inputting the input data matrix into the pre-built time-domain sparse autoencoder of the cascaded sparse autoencoder structure, the time-domain sparse autoencoder outputs a coding result of feature dimensionality reduction; According to the input and output of the time-domain sparse autoencoder, a preset loss function is used to calculate the loss value, and the connection weights between the network nodes of the time-domain sparse autoencoder are adjusted and updated; the iteration is repeated until the loss value of the time-domain sparse autoencoder no longer decreases, or a preset number of iterations is reached, thereby obtaining a trained time-domain sparse autoencoder; Input the input data matrix into the trained time domain sparse autoencoder to obtain the time domain output ; The time domain output As the input of the frequency domain sparse autoencoder, the spatiotemporal features extracted by the output of the frequency domain sparse autoencoder are used, and according to the input and output of the frequency domain sparse autoencoder, a preset loss function is used for iterative training until the loss value of the frequency domain sparse autoencoder no longer decreases, or a preset number of iterations is reached, thereby obtaining a trained frequency domain sparse autoencoder; The time domain output Input into the trained frequency domain sparse autoencoder to get the frequency domain output ; The frequency domain output As input to a softmax classifier, the softmax classifier outputs the probability of each spatial motion stimulus angle; based on the input and output of the softmax classifier, iterative training is performed using a preset loss function until the loss value of the softmax classifier no longer decreases, or a preset number of iterations is reached, thereby obtaining a trained softmax classifier; After the time domain sparse autoencoder, the frequency domain sparse autoencoder and the softmax classifier are trained, a trained cascade sparse autoencoder model is obtained.
[0010] Preferably, The expression of the preset loss function is:
[0011] Where, The term is a sparsity penalty term to limit some hidden layer neurons from being in an inactive state. is the penalty term coefficient, is the input data vector The feature encoding representation in the hidden layer is To output data, is the number of neurons in the hidden layer, is the network connection matrix, is the regularization parameter, for any hidden neuron , its sparsity is defined by KL divergence, which is defined as middle, is the average activation value of the output of the corresponding hidden layer neurons for all training trials, is a given sparsity parameter.
[0012] According to a second aspect of an embodiment of the present invention, a motor area neural signal decoding device is provided, the device comprising: Training data acquisition module: used to collect EEG signals from multiple subjects and perform signal preprocessing on the collected EEG signals; Training data classification module: used to classify the pre-processed EEG signals into 8 categories according to the direction of visual motion stimulation, and obtain the average response signal of each category; Input data acquisition module: used to obtain the average reaction data matrix of each category based on the average reaction signal of each category; splicing the 8 average reaction data matrices by column to obtain the input data matrix; Model training module: used to input the input data matrix into a pre-built cascaded sparse autoencoder structure, extract the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, iteratively train the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model; Decoding prediction module: used to convert the EEG signal to be decoded into an input data matrix and input it into the trained cascaded sparse autoencoder model, extract the spatiotemporal features of the input EEG signal through the trained cascaded sparse autoencoder model, and output the prediction result of the motion stimulation direction based on the extracted spatiotemporal features.
[0013] According to a third aspect of an embodiment of the present invention, a storage medium is provided, which stores a computer program. When the computer program is executed by a main controller, it implements each step of the logistics equipment redesign method based on digital twins.
[0014] The technical solutions provided by the embodiments of the present invention may have the following beneficial effects: The present application collects EEG signals from multiple subjects and preprocesses them, divides the preprocessed EEG signals into 8 categories according to the direction of visual motion stimulation, and obtains the average response signal of each category; obtains the average response data matrix of each category according to the average response signal of each category, splices the 8 average response data matrices by column to obtain an input data matrix; inputs the input data matrix into a pre-built cascaded sparse autoencoder structure, extracts the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, iteratively trains the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stop condition is met, and obtains a trained cascaded sparse autoencoder structure. Encoder model; convert the EEG signal to be decoded into an input data matrix and input it into a trained cascaded sparse autoencoder model, extract the spatiotemporal features of the input EEG signal through the trained cascaded sparse autoencoder model, and output the prediction result of the motion stimulation direction based on the extracted spatiotemporal features; the solution of the present application uses the cascaded sparse autoencoder model to extract the time features that change over time in the neural signal response of each trial when extracting the group EEG signal features. Therefore, the present application achieves optimal decoding performance by extracting sparsely represented time and space features, and shows the advantages of robustness, low storage and low computational burden when extracting the direction of spatial motion stimulation from the neural signal of the motor area.
[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0017] Figure 1 is a flowchart illustrating a method for decoding motor area neural signals according to an exemplary embodiment; Figure 2 is a schematic diagram showing the layout of EEG cap electrodes when collecting EEG signals from a subject according to another exemplary embodiment; Figure 3 is a schematic diagram of a framework of a cascaded sparse autoencoder structure according to another exemplary embodiment; Figure 4 is a system schematic diagram of a motor area neural signal decoding device according to another exemplary embodiment; In the accompanying figure: 1-training data acquisition module, 2-training data classification module, 3-input data acquisition module, 4-model training module, 5-decoding prediction module. DETAILED DESCRIPTION
[0018] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.
[0019] Example 1 Figure 1 FIG. 1 is a flow chart showing a method for decoding motor area neural signals according to an exemplary embodiment. Figure 1 As shown, the method includes: S1, collect EEG signals from multiple subjects and perform signal preprocessing on the collected EEG signals; S2, divide the preprocessed EEG signals into 8 categories according to the direction of visual motion stimulation, and obtain the average response signal of each category; S3, obtaining an average reaction data matrix for each category based on the average reaction signal of each category; concatenating the eight average reaction data matrices by column to obtain an input data matrix; S4, inputting the input data matrix into a pre-built cascaded sparse autoencoder structure, extracting the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, and iteratively training the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model; S5, converting the EEG signal to be decoded into an input data matrix and inputting the matrix into the trained cascaded sparse autoencoder model, extracting the spatiotemporal features of the input EEG signal through the trained cascaded sparse autoencoder model, and outputting a prediction result of the motion stimulus direction based on the extracted spatiotemporal features; It is understood that the solution of this application specifically includes: EEG signal acquisition: Five subjects were asked to wear an EEG cap containing multiple electrodes. The positions and numbers of electrodes were arranged according to the international 10-20 system, as shown in the attached figure. Figure 2 As shown, EEG signals from different brain regions are comprehensively collected; while the subjects are undergoing a visual motion stimulation task, EEG signals are continuously collected at a certain sampling frequency (e.g., 500 Hz) using an EEG acquisition device. The visual motion stimulation task may include observer displacement stimulation in front, back, left, or right directions in a virtual space. Signal preprocessing: Filtering: Use a bandpass filter to filter the collected EEG signals to remove high-frequency noise and low-frequency drift. For example, set the bandpass range to 0.5-100 Hz to retain the main frequency components related to brain activity; Artifact removal: Use methods such as independent component analysis (ICA) to remove artifacts such as electrooculography and electromyography to improve the quality of EEG signals; Z-normalization: Z-normalization is performed on the EEG signals after filtering and artifact removal to make the EEG signals of different subjects or different time periods comparable; Cascaded sparse autoencoder model training: The cascaded sparse autoencoder (SSAE) structure of this embodiment consists of three parts: a cascaded time domain / spatial domain sparse autoencoder and a softmax decoder, as shown in the attached figure. Figure 3 As shown in FIG, the SSAE network structure proposed in this embodiment consists of two three-layer SAE networks (time domain / spatial domain SAE) and a softmax classifier. The training of the three parts of the network is carried out in sequence (SAE1→SAE2→softmax classifier), and the back propagation error is limited to each part. is the data matrix The data of one trial in is used as an input vector for SAE1. is the reconstructed signal vector of the input vector at the output layer of SAE1, and XN is the group response matrix The data of a trial is composed of the eigenvalues of the neural signal channels of each motor area encoded by SAE1 at the same spatial stimulation angle. The corresponding data matrix One of the trials is the feature vector of XN after SAE2 encoding.
[0020] The specific training process is as follows: Time-domain sparse autoencoder training: The input layer of the time-domain sparse autoencoder is set with 1200 artificial neurons, the output layer is set with 1200 artificial neurons, and the number of hidden layer neurons of the artificial neural network is set to 1. During the training phase, the EEG signals of each motor area neural response training set trial are z-normalized and then divided into 8 categories according to the direction of visual motion stimulation. The average response signal of the training set data of each category is obtained. The average response matrix of the group motor area neural signal for spatial motion stimulation in each direction category is defined as (Matrix size: OK, columns), where is the number of nerve response channels in the motor area, is the time domain data length of the motor area EEG signal, is the visual motion stimulus direction category. Subsequently, the 8 average response data matrices are spliced by column as the training input data matrix of the time-domain sparse autoencoder. During training, the connection weights between network nodes are adjusted and updated through the preset loss function. The expression of the preset loss function is as follows:
[0021] Where, The term is a sparsity penalty term to limit some hidden layer neurons from being in an inactive state. is the penalty term coefficient, is the input data vector The feature encoding representation in the hidden layer is To output data, is the number of neurons in the hidden layer, is the network connection matrix, is the regularization parameter, for any hidden neuron , its sparsity is defined by KL divergence, which is defined as middle, is the average activation value of the output of the corresponding hidden layer neurons for all training trials, Given a sparsity parameter, the training is repeated until the loss value of the loss function no longer decreases, or the preset number of iterations is reached, and a trained time-domain sparse autoencoder is obtained. After the time domain sparse autoencoder is trained, the training set data matrix of the neural signal of each motor area ( ) is input into the network to obtain the encoding result of feature dimensionality reduction ( ) is used for the next step of spatial sparse autoencoder training; Spatial Domain Sparse Autoencoder Training: Data Matrix It is used for training spatial domain SAE. The number of nodes in the input layer and output layer of the network structure is , which is equal to the number of EEG signal channels in the motor area, and the number of hidden layer nodes is The loss function used in the training process is consistent with the loss function of the time domain sparse autoencoder training. The stopping condition of the training is also that the loss value no longer decreases or reaches the preset number of iterations. After the spatial domain sparse autoencoder training is completed, Input the network and obtain the activation value matrix of the hidden layer nodes of the network , that is, completing the entire spatiotemporal feature extraction process; Softmax classifier training: data matrix As the training input data of the softmax classifier in the third part, the above loss function is also used for iterative training. After the softmax classifier training is completed, the classifier gives the test set The probability that each trial sample corresponds to each spatial motion stimulus angle is expressed as follows:
[0022] Where, and They represent the activation values of the output layer of the softmax classifier respectively. The stimulation angle corresponding to the maximum probability is the predicted value of the SSAE for the motion stimulation angle of this trial.
[0023] This embodiment also discloses a comparative test case of the model of this embodiment and the prior art, which is as follows: The training of the designed SSAE structure is carried out from time domain SAE and spatial domain SAE to softmax classifier in sequence. The traditional gradient descent algorithm is used in the error back propagation algorithm for SSAE network weight adjustment. The maximum number of training rounds is set to 2200 times. The loss function of SSAE network is Regularization parameter Set to 0.004, the network sparsity penalty weight parameter Set to 4, for each motor area neural signal channel, randomly obtain 20 trial response data under each motion stimulus direction condition, and randomly divide them into training set and test set, of which 75% of the data is the training set and 25% of the data is the test set ( , ), using the training set data to train the cascaded sparse autoencoder network structure according to the above method, after the training is completed, the test set data is input into the network structure to obtain the discrimination results, in order to obtain the probability distribution characteristics of the accuracy of the multi-channel decoding of the final motion stimulus direction. The above SSAE network training and testing process is repeated 80 times; To verify the effectiveness of the SSAE model, this embodiment compares it with other commonly used group neural signal encoding and decoding methods, using the same data set and the same test set and training set division method, and repeating the model training and testing 80 times; this embodiment compares the group decoding performance with group decoding methods using (1) traditional support vector machine (SVM), (2) sparse feature representation method, and (3) deep learning artificial neural network method.
[0024] First, this embodiment uses Gaussian kernel SVM to decode the motion stimulus direction information in the motor area neural signal, and then normalizes the single trial response of the channel in each motor area neural signal to the motion stimulus presentation stage in each direction, and after time domain averaging, finally splices them into a single trial group response vector, and uses it as the input vector of the SVM model. The training and testing of the SVM model are completed using the LIBSVM toolbox, and the Gaussian kernel function and slack variables are used to realize the soft margin SVM classifier. The training process uses 4-fold cross validation and grid search method to optimize the model penalty parameters and Gaussian kernel function parameters, and the optimization range is [2 -10 , 2 -9 , 2 -8 … 2 8 , 2 9 , 2 10 ]; Then, this embodiment uses three types of sparse feature representation methods to perform the same group response decoding process, of which the first type is the sparse feature representation classifier (sparse representation classification, SRC), which combines the dictionary learning method and the sparse feature representation method. Each motion stimulus direction category has its own dictionary, and then finds the category with the smallest dictionary and signal residual under the sparse restriction condition, that is, completes the classification judgment. In this embodiment, the restriction parameter of the dictionary learning of this method is set to 0.001. The second type is the Fisher classification dictionary learning method (Fisher discrimination dictionary learning, FDDL), which is designed based on the Fisher classification criterion and is a supervised dictionary learning method commonly used in sparse feature representation. This method constructs a structured dictionary based on the classification label and has good classification ability. In this embodiment, the scalar parameter (scalar parameter) of the correlation coefficient matrix and the discrimination constraint parameter (discrimination constraint parameter) are used. The third category is the LC-KSVD method. In addition to using the class labels of the training data, this method also associates label information with each dictionary item to enhance the discriminability of sparse coding during the dictionary learning process. This method introduces a new label consistency constraint and combines it with the reconstruction error and classification error to form a unified objective function. It jointly learns an overcomplete dictionary and an optimal linear classifier. Its sparsity threshold is set to 10, and the label weight and classification error weight are both 0.001.
[0025] Finally, this embodiment also compares the feature extraction effect of the long short-term memory recursive network structure based on the deep artificial neural network. LSTM maintains the error flow by introducing a memory unit structure (including an input gate, an output gate, and a forget gate), which can avoid the problem of error signal disappearance in the backward propagation of the recurrent network. The number of hidden layer nodes in the network is set to 128, and the dropout probability is set to 0.5 to prevent overfitting.
[0026] Through comparison of experimental results, it was found that compared with other commonly used decoding methods, the SSAE method proposed in this embodiment has a higher decoding accuracy and relatively small computational and storage burden. Many commonly used encoding and decoding methods do not take into account the temporal dynamic characteristics of the data set. As mentioned above, the motor area neural signals have strong spatiotemporal dynamic characteristics, which have an important impact on the model encoding and decoding process. For example, the accuracy of the traditional SVM decoding method is lower than that of other methods. This is because the method ignores the temporal characteristics of the data and only uses a few support vectors to determine the decision boundary, which is likely to ignore the essential characteristics of the high-dimensional neural signal response data. In some group neural signal encoding and decoding studies similar to the present application, the decoding accuracy of this method is also low. In addition, for the SVM method, the accuracy of using motor area neural signals with direction selectivity for encoding and decoding is significantly higher than that of using all recorded motor area neural signals (p<0.05), indicating that this method has poor robustness and is easily affected by noise.
[0027] Compared with SRM (FDDL, LC-KSVD and SRC), the SSAE model uses time-domain SAE to extract the time-varying temporal features in the neural signal response of each trial. Therefore, SSAE not only models the spatial domain dynamics in the dataset, but also considers the temporal characteristics of the data. By extracting sparsely represented temporal and spatial features and using a softmax classifier, SSAE achieves optimal decoding performance and shows the advantages of robustness, low storage and low computational burden when extracting the direction of spatial motion stimuli from the neural signals of the motor area.
[0028] Although the LSTM model can model the temporal dynamics of time series data, its decoding accuracy is still lower than that of the SSAE model. One possible explanation is that the smaller amount of neural signal data in the motor area leads to underfitting of the LSTM network training, making it more susceptible to noise in the time series data. Furthermore, the SSAE network has fewer connection weights than the LSTM network, making network training easier to converge. In summary, especially during online decoding, it is necessary to balance the complexity of various encoding and decoding methods, the amount of data required, and the decoding accuracy to select the appropriate decoding method.
[0029] Example 2 Figure 4is a system schematic diagram of a motor area neural signal decoding device according to another exemplary embodiment, the device comprising: Training data acquisition module 1: used to collect EEG signals from multiple subjects and perform signal preprocessing on the collected EEG signals; Training data classification module 2: used to classify the pre-processed EEG signals into 8 categories according to the direction of visual motion stimulation and obtain the average response signal of each category; Input data acquisition module 3: used to obtain the average reaction data matrix of each category based on the average reaction signal of each category; splicing the 8 average reaction data matrices by column to obtain the input data matrix; Model training module 4: used to input the input data matrix into a pre-built cascaded sparse autoencoder structure, extract the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, iteratively train the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model; Decoding prediction module 5: used to convert the EEG signal to be decoded into an input data matrix and input it into the trained cascaded sparse autoencoder model, extract the spatiotemporal features of the input EEG signal through the trained cascaded sparse autoencoder model, and output the prediction result of the motion stimulation direction based on the extracted spatiotemporal features.
[0030] Example 3: This embodiment provides a storage medium storing a computer program. When the computer program is executed by a host controller, each step of the above method is implemented. It is understandable that the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.
[0031] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.
[0032] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" is at least two.
[0033] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.
[0034] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.
[0035] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.
[0036] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
[0037] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.
[0038] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
[0039] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A method for decoding motor area neural signals, characterized in that: The method comprises: Collecting EEG signals from multiple subjects and performing signal preprocessing on the collected EEG signals; The preprocessed EEG signals were divided into 8 categories according to the direction of visual motion stimulation, and the average response signal of each category was obtained; Obtain the average response data matrix of each category based on the average response signal of each category; concatenate the eight average response data matrices by column to obtain the input data matrix; Inputting the input data matrix into a pre-built cascaded sparse autoencoder structure, extracting the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, and iteratively training the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model; The EEG signal to be decoded is converted into an input data matrix and input into the trained cascaded sparse autoencoder model. The spatiotemporal features of the input data matrix are extracted by the trained cascaded sparse autoencoder model, and the prediction result of the motion stimulation direction is output based on the extracted spatiotemporal features.
2. The method according to claim 1, characterized in that The collecting of EEG signals of multiple subjects includes: Having multiple subjects wear an EEG cap containing multiple electrodes, where the position and number of electrodes of the EEG cap meet preset standards; A plurality of subjects were asked to accept a visual motion stimulation task, and EEG signals were continuously collected at a certain sampling frequency. The visual motion stimulation task included observer displacement stimulation in front, back, left, and right directions in a virtual space.
3. The method according to claim 2, characterized in that The signal preprocessing of the collected EEG signal includes: The collected EEG signal is filtered using a bandpass filter to remove high-frequency noise and low-frequency drift, wherein the bandpass range of the bandpass filter is 0.5-100 Hz; The independent component analysis method is used to remove artifact interference from the filtered EEG signal; The EEG signals after artifact interference removal were z-normalized to make the EEG signals of different subjects or different time periods comparable.
4. The method according to claim 3, characterized in that The pre-built cascaded sparse autoencoder structure consists of three parts: a cascaded time-domain sparse autoencoder, a frequency-domain sparse autoencoder, and a classifier; The input layer of the time-domain sparse autoencoder is provided with X artificial neurons, the output layer is provided with X artificial neurons, and the number of neurons in the hidden layer is 1; The number of nodes in the input layer and output layer of the frequency domain sparse autoencoder is set to N , which is equal to the number of EEG signal channels in the motor area, sets the number of hidden layer nodes after debugging; The classifier is a softmax classifier.
5. The method according to claim 4, characterized in that The step of inputting the input data matrix into a pre-built cascaded sparse autoencoder structure, extracting the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, and iteratively training the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model, comprising: Inputting the input data matrix into the pre-built time-domain sparse autoencoder of the cascaded sparse autoencoder structure, the time-domain sparse autoencoder outputs a coding result of feature dimensionality reduction; According to the input and output of the time-domain sparse autoencoder, a preset loss function is used to calculate the loss value, and the connection weights between the network nodes of the time-domain sparse autoencoder are adjusted and updated; the iteration is repeated until the loss value of the time-domain sparse autoencoder no longer decreases, or a preset number of iterations is reached, thereby obtaining a trained time-domain sparse autoencoder; Input the input data matrix into the trained time domain sparse autoencoder to obtain the time domain output ; The time domain output As the input of the frequency domain sparse autoencoder, the spatiotemporal features extracted by the output of the frequency domain sparse autoencoder are used, and according to the input and output of the frequency domain sparse autoencoder, a preset loss function is used to perform iterative training until the loss value of the frequency domain sparse autoencoder no longer decreases, or a preset number of iterations is reached, thereby obtaining a trained frequency domain sparse autoencoder; The time domain output Input into the trained frequency domain sparse autoencoder to get the frequency domain output ; The frequency domain output As input to a softmax classifier, the softmax classifier outputs the probability of each spatial motion stimulus angle; based on the input and output of the softmax classifier, iterative training is performed using a preset loss function until the loss value of the softmax classifier no longer decreases, or a preset number of iterations is reached, thereby obtaining a trained softmax classifier; After the time domain sparse autoencoder, the frequency domain sparse autoencoder and the softmax classifier are trained, a trained cascade sparse autoencoder model is obtained.
6. The method according to claim 5, characterized in that The expression of the preset loss function is: Where, The term is a sparsity penalty term to limit some hidden layer neurons from being in an inactive state. is the penalty term coefficient, is the input data vector The feature encoding representation in the hidden layer is To output data, is the number of neurons in the hidden layer, is the network connection matrix, is the regularization parameter, for any hidden neuron , its sparsity is defined by KL divergence, which is defined as middle, is the average activation value of the output of the corresponding hidden layer neurons for all training trials, is a given sparsity parameter.
7. A motor area neural signal decoding device, characterized in that: The device comprises: Training data acquisition module: used to collect EEG signals from multiple subjects and perform signal preprocessing on the collected EEG signals; Training data classification module: used to classify the pre-processed EEG signals into 8 categories according to the direction of visual motion stimulation, and obtain the average response signal of each category; Input data acquisition module: used to obtain the average reaction data matrix of each category based on the average reaction signal of each category; splicing the 8 average reaction data matrices by column to obtain the input data matrix; Model training module: used to input the input data matrix into a pre-built cascaded sparse autoencoder structure, extract the spatiotemporal features of the input data matrix through the pre-built cascaded sparse autoencoder structure, iteratively train the pre-built cascaded sparse autoencoder structure according to a preset loss function until a preset stopping condition is met, thereby obtaining a trained cascaded sparse autoencoder model; Decoding prediction module: used to convert the EEG signal to be decoded into an input data matrix and input it into the trained cascaded sparse autoencoder model, extract the spatiotemporal features of the input EEG signal through the trained cascaded sparse autoencoder model, and output the prediction result of the motion stimulation direction based on the extracted spatiotemporal features.
8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the main controller, each step of the motor area neural signal decoding method according to any one of claims 1 to 6 is implemented.