A method and system for analyzing movement intention patterns

Abstract

The application discloses a motion intention mode analysis method and system, and relates to the field of electroencephalogram signal processing. The method comprises the following steps: acquiring electroencephalogram signals of a user before and after various motion modes, and pre-processing the electroencephalogram signals; calibrating the pre-processed electroencephalogram signals for a set time length; determining source-reconstructed electroencephalogram signals by using a signal source reconstruction method according to the calibrated electroencephalogram signals; performing sliding time window extraction processing on the source-reconstructed electroencephalogram signals to obtain a first sample set; training a decoder by using the first sample set to obtain a decoder based on a time mechanism; performing slicing processing and zeroing processing on the source-reconstructed electroencephalogram signals to obtain a second sample set; training the decoder by using the second sample set to obtain a decoder based on a space mechanism; and performing motion intention mode analysis by using the decoder based on the time mechanism and the decoder based on the space mechanism. The application can obtain time mechanisms and space mechanisms of various motion intention modes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of electroencephalogram signal processing, in particular to a motor intention pattern analysis method and system. BACKGROUND

[0002] The existing method records the traces of brain activity through electroencephalogram and other brain signal acquisition techniques, and then explores the brain regularity through the method of establishing a brain model and statistics. This exploration is usually a macroscopic regularity, and it is difficult to explore the mechanism of a movement pattern with high time resolution or high action similarity. Therefore, expensive instruments are usually used to solve these problems. SUMMARY

[0003] The purpose of the present application is to provide a motor intention pattern analysis method and system, which can obtain the time mechanism and spatial mechanism of various motor intention patterns.

[0004] To achieve the above purpose, the present application provides the following scheme:

[0005] A motor intention pattern analysis method, comprising:

[0006] Obtaining electroencephalogram signals of a user before and after various movement patterns, and preprocessing the electroencephalogram signals; the preprocessing includes: polar positioning, removing useless electrodes, re-referencing, filtering, segmenting, and baseline correction;

[0007] Calibrating the electroencephalogram signals preprocessed for a set time length;

[0008] According to the calibrated electroencephalogram signals, determining the electroencephalogram signals after source reconstruction by using a signal source reconstruction method;

[0009] Performing sliding time window extraction processing on the electroencephalogram signals after source reconstruction to obtain a first sample set; and training a decoder using the first sample set to obtain a decoder based on a time mechanism;

[0010] Performing slicing processing and zeroing processing on the electroencephalogram signals after source reconstruction to obtain a second sample set; and training a decoder using the second sample set to obtain a decoder based on a spatial mechanism;

[0011] Using the decoder based on the time mechanism and the decoder based on the spatial mechanism to perform motor intention pattern analysis.

[0012] Optionally, the device for obtaining the electroencephalogram signals of the user before and after various movement patterns is an electroencephalogram cap device or a near-infrared device.

[0013] Optionally, the calibration of the electroencephalogram signals preprocessed for a set time length specifically includes:

[0014] record the angle change of the head throughout the whole process by using the IMU angle sensor;

[0015] extract the electroencephalogram signals of the set duration according to the time point at which the angle change is greater than the change threshold.

[0016] Optionally, the signal source reconstruction method comprises a linearly constrained minimum variance method or a dSPM method.

[0017] Optionally, the decoder comprises a CNN convolutional layer, two BiLSTM layers, and one fully connected layer.

[0018] Optionally, the source-reconstructed electroencephalogram signals are subjected to slicing processing and zeroing processing to obtain a second sample set; and the decoder is trained by using the second sample set to obtain a decoder based on a spatial mechanism, specifically comprising:

[0019] The source-reconstructed electroencephalogram signals are subjected to slicing processing;

[0020] The channels are subjected to zeroing processing to obtain samples with channel information differences;

[0021] The second sample set is determined according to the samples with channel information differences and the corresponding sliced electroencephalogram signals.

[0022] A motion intention pattern analysis system comprises:

[0023] A preprocessing module is configured to acquire electroencephalogram signals before and after various motion patterns of a user and to preprocess the electroencephalogram signals; the preprocessing comprises polar orientation, rejection of useless electrodes, re-reference, filtering, segmentation, and baseline correction.

[0024] A calibration module is configured to calibrate electroencephalogram signals of a set duration after preprocessing.

[0025] According to the calibrated electroencephalogram signals, a signal source reconstruction method is used to determine source-reconstructed electroencephalogram signals.

[0026] A decoder determination module based on a time mechanism is configured to perform sliding time window extraction processing on the source-reconstructed electroencephalogram signals to obtain a first sample set; and the decoder is trained by using the first sample set to obtain a decoder based on a time mechanism.

[0027] A decoder determination module based on a spatial mechanism is configured to perform slicing processing and zeroing processing on the source-reconstructed electroencephalogram signals to obtain a second sample set; and the decoder is trained by using the second sample set to obtain a decoder based on a spatial mechanism.

[0028] A motion intention pattern analysis module is configured to perform motion intention pattern analysis by using the decoder based on the time mechanism and the decoder based on the spatial mechanism.

[0029] An electronic device includes a memory for storing a computer program and a processor for running the computer program to cause the electronic device to perform the motion intention pattern analysis method.

[0030] According to the specific embodiments of the present application, the following technical effects are disclosed.

[0031] The motion intention pattern analysis method and system provided by the present application can explore the brain physiological mechanism by establishing a decoder of a deep learning model. The sample set with subtle differences in time or space characteristics is obtained through special processing of the sample, and then the results of these subtle input differences, i.e., the obvious difference in accuracy, are obtained through model training and sample evaluation to explore the regularity of certain brain physiological mechanism in time or space. The present application can obtain the differences in brain activity under high time and space resolution and the physiological mechanism behind these differences through deep learning without using expensive instruments. The present application obtains high-density signals of the cerebral cortex through source reconstruction, thereby obtaining the basic signals for physiological mechanism exploration. The deep learning algorithm with time and space characteristic extraction of the electroencephalogram signal decoding is designed to provide the decoder for subsequent model training. The model with time regularity exploration function is obtained through the sliding time window processing to obtain the sample with higher time resolution difference. The model with space regularity exploration function is obtained through the channel zero processing to obtain the sample with channel information difference. In summary, the motion intention pattern analysis is realized. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0033] Figure 1 The flowchart of the motion intention pattern analysis method provided by the present application is shown. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0035] The application aims to provide a motion intention mode analysis method and system, which can obtain time mechanism and space mechanism of various motion intention modes.

[0036] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the application will be further described in detail below in combination with the drawings and specific embodiments.

[0037] As shown in the drawings, Figure 1 The motion intention mode analysis method provided by the application comprises the following steps.

[0038] S101, acquiring electroencephalogram signals of a user before and after various motion modes, and pre-processing the electroencephalogram signals; the pre-processing comprises electrode positioning, removing useless electrodes, re-referencing, filtering, segmenting and baseline correction.

[0039] The total number of acquired electroencephalogram signal channels is 64, and only 59 of them actually contain useful information. The actual channels correspond to the voltage size measured by the electrodes at a certain position on the brain.

[0040] As a specific embodiment, the device for acquiring the electroencephalogram signals of the user before and after various motion modes is an electroencephalogram cap device or a near-infrared device.

[0041] The specific process of pre-processing is as follows.

[0042] 1. Electrode positioning: a channel position information matching the recording data is loaded to perform spatial channel registration, and each channel name is checked and aligned with a spatial position.

[0043] 2. Removing useless electrodes: some channels that are not needed and do not contain information are removed.

[0044] 3. Re-referencing: the average voltage of bilateral papillae is used as a reference, and the data of each channel is subtracted from it to obtain the relative value of each channel.

[0045] 4. Filtering: a 1-40Hz band-pass filter is used for filtering.

[0046] 5. Segmentation: the electroencephalogram data 2s before each action required in the entire experiment is segmented, and other data is removed.

[0047] 6. Baseline correction: the average value of the first part of the electroencephalogram data is used as a reference value, and the subsequent electroencephalogram data is subtracted from the reference value.

[0048] S102, calibrating the pre-processed electroencephalogram signals of a set time length.

[0049] S103, determining the source-reconstructed electroencephalogram signals by using a signal source reconstruction method according to the calibrated electroencephalogram signals.

[0050] S103 specifically comprises:

[0051] The data is sent to the computer through a TCP-IP communication mode.

[0052] The angle change of the head during the whole process is recorded by using the IMU angle sensor.

[0053] The EEG signal of a set time is extracted according to the time point at which the angle change is greater than the change threshold.

[0054] As a specific embodiment, the signal source reconstruction method comprises a linearly constrained minimum variance (LCMV) method or a dSPM method.

[0055] As a specific embodiment, the calibration method requires that the effective signal within 2 seconds before the occurrence of various motion actions be reserved for subsequent physiological mechanism analysis of the time period.

[0056] Specifically, the EEG signals of each channel on the scalp surface are projected into each point on the cerebral cortex surface by the signal source reconstruction method, i.e., the linearly constrained minimum variance method (LCMV). After source reconstruction, the deep signals inside the brain can be obtained, which facilitates the research on the brain physiological mechanism. After source reconstruction, the number of channels of the EEG signals will be greatly improved, and better spatial accuracy will be obtained.

[0057] S104, the EEG signals after source reconstruction are subjected to sliding time window extraction processing to obtain a first sample set; and the first sample set is used to train a decoder to obtain a decoder based on a time mechanism.

[0058] Specifically, the source signals within 2 seconds before the occurrence of the motion are subjected to sliding time window extraction processing, and the length of the time window is 300 ms, so that the motion intention pattern samples (the first sample set) of each time period of 300 ms length near each time point can be obtained.

[0059] The accuracy of each motion pattern in each time period is verified to obtain the change of the accuracy of each pattern. If the accuracy of the time period is high, it indicates that the brain is in a high activation state of the motion intention in the time period.

[0060] S105, the EEG signals after source reconstruction are subjected to slicing processing and zeroing processing to obtain a second sample set; and the second sample set is used to train a decoder to obtain a decoder based on a spatial mechanism.

[0061] The decoder in S104 and S105 comprises a CNN convolution layer, two BiLSTM layers, and a fully connected layer. The CNN convolution layer is used to extract spatial features of the motion intention; and the BiLSTM layer is used to extract time sequence features.

[0062] S105 specifically comprises:

[0063] The source-reconstructed electroencephalogram is sliced.

[0064] Each channel is zeroed to obtain samples with channel information differences.

[0065] A second sample set is determined according to the samples with channel information differences and the corresponding sliced electroencephalogram.

[0066] Specifically, the electroencephalogram of a time length of 2s before movement is sliced to obtain an electroencephalogram of a certain short time period, and then each channel is zeroed to obtain samples (second sample set) with channel information differences.

[0067] According to the accuracy of the verification set by the decoder based on the spatial mechanism, the key degree distribution of the channel accuracy of the time period can be obtained, so it can be known that certain channels play a key role in the brain movement intention and the region is in a highly activated state.

[0068] S106, using the decoder based on the time mechanism and the decoder based on the spatial mechanism to analyze the movement intention mode.

[0069] As another specific embodiment, the application also provides a movement intention mode analysis system, comprising:

[0070] A preprocessing module is configured to acquire electroencephalograms of a user before and after various movement modes and to preprocess the electroencephalograms; the preprocessing comprises polar orientation, rejection of useless electrodes, re-reference, filtering, segmentation, and baseline correction.

[0071] A calibration module is configured to calibrate electroencephalograms of a set time length after preprocessing.

[0072] According to the calibrated electroencephalograms, a signal source reconstruction method is used to determine source-reconstructed electroencephalograms.

[0073] A decoder based on the time mechanism determination module is configured to perform sliding time window extraction processing on the source-reconstructed electroencephalograms to obtain a first sample set; and train a decoder using the first sample set to obtain a decoder based on the time mechanism.

[0074] A decoder based on the spatial mechanism determination module is configured to perform slicing and zeroing processing on the source-reconstructed electroencephalograms to obtain a second sample set; and train a decoder using the second sample set to obtain a decoder based on the spatial mechanism.

[0075] A movement intention mode analysis module is configured to use the decoder based on the time mechanism and the decoder based on the spatial mechanism to analyze the movement intention mode.

[0076] As another specific embodiment, the present application also provides an electronic device, comprising a memory for storing a computer program and a processor for running the computer program to make the electronic device execute the motion intention pattern analysis method.

[0077] The various embodiments are described in a progressive manner in the specification, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0078] The principles and implementation manners of the present application are described by using specific examples in the specification. The above embodiment descriptions are only used to help understand the method of the present application and its core idea. Meanwhile, for the general technical personnel in the field, the specific implementation manners and application ranges can be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A method of motion intention pattern analysis, characterized by, The method comprises the following steps: acquiring electroencephalogram signals of a user before and after various movement modes, and preprocessing the electroencephalogram signals; the preprocessing comprises electrode positioning, removing useless electrodes, re-referencing, filtering, segmenting and baseline correction; calibrating the electroencephalogram signals preprocessed for a set time length; determining source-reconstructed electroencephalogram signals by using a signal source reconstruction method according to the calibrated electroencephalogram signals; performing sliding time window extraction processing on the source-reconstructed electroencephalogram signals to obtain a first sample set; and training a decoder by using the first sample set to obtain a decoder based on a time mechanism; performing slicing processing and zero processing on the source-reconstructed electroencephalogram signals to obtain a second sample set; and training the decoder by using the second sample set to obtain a decoder based on a space mechanism; performing movement intention mode analysis by using the decoder based on the time mechanism and the decoder based on the space mechanism; the decoder comprises a CNN convolutional layer, two BiLSTM layers and one fully connected layer; the slicing processing and zero processing on the source-reconstructed electroencephalogram signals to obtain the second sample set; and the training of the decoder by using the second sample set to obtain the decoder based on the space mechanism specifically comprise: performing slicing processing on the source-reconstructed electroencephalogram signals; performing zero processing on each channel one by one to obtain samples with channel information differences; determining the second sample set according to the samples with channel information differences and corresponding electroencephalogram signals after the slicing processing.

2. The method of claim 1, wherein The device for acquiring the electroencephalogram signals of the user before and after various movement modes is an electroencephalogram cap device.

3. The method of claim 1, wherein The calibration of the electroencephalogram signals preprocessed for a set time length specifically comprises: recording angle changes of the head in the whole process by using an IMU angle sensor; extracting electroencephalogram signals of a set time length according to time points at which the angle changes are greater than a change threshold.

4. The method of claim 1, wherein, The signal source reconstruction method comprises a linearly constrained minimum variance method.

5. A motion intention pattern analysis system for implementing the motion intention pattern analysis method according to any one of claims 1 to 4, characterized by The method comprises the following steps: a preprocessing module is configured to acquire electroencephalogram signals of a user before and after various movement modes, and preprocess the electroencephalogram signals; the preprocessing comprises electrode positioning, removing useless electrodes, re-referencing, filtering, segmenting and baseline correction; a calibration module is configured to calibrate electroencephalogram signals preprocessed for a set time length; a source-reconstructed electroencephalogram signal is determined by using a signal source reconstruction method according to the calibrated electroencephalogram signals; a decoder based on a time mechanism is determined by performing sliding time window extraction processing on the source-reconstructed electroencephalogram signals to obtain a first sample set; and the decoder is trained by using the first sample set to obtain the decoder based on the time mechanism; a decoder based on a space mechanism is determined by performing slicing processing and zero processing on the source-reconstructed electroencephalogram signals to obtain a second sample set; and the decoder is trained by using the second sample set to obtain the decoder based on the space mechanism; a movement intention mode analysis module is configured to perform movement intention mode analysis by using the decoder based on the time mechanism and the decoder based on the space mechanism.

6. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to perform a movement intention mode analysis method according to any one of claims 1 to 4.

Images