An adaptive algorithm for cross-subject motor imagery classification

Abstract

The application discloses an adaptive algorithm for cross-user motor imagery classification, and relates to the field of electroencephalogram data classification, and comprises the following steps: S1, constructing an electrode distance-based inter-electrode loss weight function; S2, constructing an electrode distance-based inter-electrode data distribution difference loss function; S3, constructing a network overall structure, and minimizing the inter-electrode loss weight function and the data distribution difference loss function through training. The method can achieve higher accuracy in the same electroencephalogram data classification task.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of electroencephalogram data classification, and more particularly, to an adaptive algorithm for cross-user motor imagery classification. BACKGROUND

[0002] The human brain generates bioelectricity when working, which causes the scalp surface potential to change over time. This change can be recorded by a brain-computer interface (BCI) to obtain an electroencephalogram (EEG).

[0003] The electroencephalogram records the spatiotemporal changes of the scalp surface potential and can be used to describe and analyze various different brain activities, and has been widely used in the fields of brain science, medicine, etc. Among various electroencephalogram signals, motor imagery (MI) signals based on the electroencephalogram, i.e., motor imagery electroencephalogram signals, can be used to control many external devices such as wheelchairs and mechanical arms, and have become a research hotspot in recent years.

[0004] For different users, motor imagery electroencephalogram signal data often has large differences. This is because: first, different users have different head shapes (head circumference, scalp shape); second, the brain region activation and excitation degree of different users are different when performing the same motor imagery task; and third, the collection of electroencephalogram data is easily disturbed by environmental noise, user fatigue, and other uncertain factors.

[0005] The collection of motor imagery electroencephalogram data is time-consuming and laborious. Therefore, more and more research has begun to focus on the classification of cross-user motor imagery electroencephalogram data in order to utilize the electroencephalogram data of different users to complete classification, thereby increasing the sample size of the training data and improving the generalization of the trained model.

[0006] However, existing research on the classification of cross-user motor imagery electroencephalogram data often only performs global domain and class-related domain adaptive alignment on the overall features and class-related features of the electroencephalogram data, and has not explored the data distribution differences between different electrodes, whether within a user or between users. SUMMARY

[0007] The technical problem to be solved by the present application is to provide a cross-user motor imagery classification deep electrode adaptive convolution network based on electrode distribution differences, which minimizes the electrode channel differences within and between users in the source domain and the target domain, and the distribution differences of the overall data of the source domain and the target domain, to complete the classification of cross-user motor imagery electroencephalogram data from a single source domain user to a single target domain user. Compared with other existing research in the same field, the method and network proposed by the present application achieve higher accuracy in the same classification task.

[0008] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0009] An adaptive algorithm for cross-subject motor imagery classification, comprising the following steps:

[0010] S1: Constructing an electrode distance-based inter-electrode loss weight function;

[0011] S2: Constructing an electrode distance-based inter-electrode data distribution difference loss function;

[0012] S3: Constructing the overall network structure, and minimizing the electrode distance-based inter-electrode loss weight function and the data distribution difference loss function through training.

[0013] Further, in S1: the electrode distance-based inter-electrode loss weight function is expressed as:

[0014]

[0015] wherein:

[0016] X is the horizontal coordinate data of the electrode channel of the data set;

[0017] Y is the vertical coordinate data of the electrode channel of the data set;

[0018] l and h are the maximum values of the horizontal and vertical coordinates of the electrode channel of the data set, respectively;

[0019] d is the distance between two electrodes.

[0020] Further, the data set is the "BCI Competition IV IIa" data set.

[0021] Further, the electrode distance-based inter-electrode data distribution difference loss function comprises the following two kinds:

[0022] The electrode distance-based inter-electrode data distribution difference loss function between users and

[0023] The electrode distance-based inter-electrode data distribution difference loss function between source domain and target domain

[0024] Further,

[0025] wherein:

[0026] and are the electrode distance-based inter-electrode loss weight functions;

[0027] is the electrode distance-based source domain data distribution difference loss function;

[0028] a data distribution difference loss function between electrode data of the target domain user based on electrode distance;

[0029] are determined by the following formulas, respectively:

[0030]

[0031] wherein:

[0032] is a loss weight function between electrodes in S1;

[0033] respectively represent the expectation of the source domain and electrode correlation, and the expectation of the target domain and electrode correlation;

[0034] E S , E T respectively represent the expectation of the source domain data and the target domain data;

[0035] e S ,e T respectively represent the number of electrode channels of the source domain data and the target domain data;

[0036] φ[·] is a nonlinear mapping function that can map features to RKHS space;

[0037] z S ,z T respectively represent the deep features of the source domain data and the target domain data extracted by the network;

[0038] n S ,n T respectively represent the sample number of the source domain data and the target domain data.

[0039] Further, the data distribution difference loss function between the source domain and the target domain user based on electrode distance is determined by the following formula:

[0040]

[0041] Further, in S3, the method for minimizing the loss weight function between electrodes and the data distribution difference loss function is a convolutional neural network training method.

[0042] Further, the overall structure of the network includes a feature extraction layer, a first fully connected layer connected to the feature extraction layer, and a second fully connected layer connected to the first fully connected layer.

[0043] Further, in S3, and the minimization sequence is: first minimize and then minimize

[0044] The advantages of the present application over the prior art are:

[0045] 1. The data distribution differences between different electrodes are explored, whether within or outside the user, and the classification of cross-user motor imagery electroencephalogram data from a single source domain user to a single target domain user is completed by respectively minimizing the electrode channel differences within and between source domain and target domain users, and the distribution differences of the overall data of source domain and target domain users, thereby improving the generalization ability of the model and the performance of cross-user classification.

[0046] 2. The distribution difference loss function related to the electrode is constructed to adaptively reduce the distribution difference between electrodes, so that the data distribution between different electrodes is more compact, and the classification difficulty of the model is reduced.

[0047] 3. The different electrode losses constructed by using 3D fully connected layers are utilized to minimize the differences between different electrode channels of users as much as possible before minimizing the distribution differences of the overall data of source domain and target domain users, and this operation of minimizing the electrode differences is first performed within a single user (within the source domain user and within the target domain user), and then between users (between the source domain user and the target domain user), so that there is no conflict and contradiction between the defined losses, and all the self-defined loss functions are better minimized. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is the overall structure of the network of the present application;

[0049] Figure 2 is the position diagram of the electrode channel of the data set of the present application;

[0050] Figure 3 is the simplified network structure diagram of the present application. DETAILED DESCRIPTION

[0051] The specific embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0052] The cross-user motor imagery classification deep electrode adaptive convolution network based on the distribution difference between electrodes proposed in the present application completes the classification of cross-user motor imagery electroencephalogram data from a single source domain user to a single target domain user by respectively minimizing the electrode channel differences within and between source domain and target domain users, and the distribution differences of the overall data of source domain and target domain users, and compared with other existing researches in the same field, the network proposed in the present application achieves higher accuracy in the same classification task, and the overall structure of the network is as follows: Figure 1 and steps 1 to 3.

[0053] 1. Construct an electrode inter-loss weight function based on electrode distance:

[0054] First, a weighting function for the loss between different electrodes needs to be constructed, as we want to incorporate more spatial information into the electrode loss. The weights are determined by the relative positions of the two electrode channels. Taking the "BCI Competition IV IIa" dataset as an example, this dataset contains 118 electrode channels and is the most commonly used dataset for studying cross-user motor imagery EEG transfer learning. A schematic diagram of the electrode channel positions in this dataset is drawn using Matlab, as shown below. Figure 2 As shown (the coordinates and corresponding numbers of the electrode channels are provided by the BCI Competition IV official website): Let the x-axis data of the electrode channels in the dataset be X, and the y-axis data be Y, then the weights between the electrode channels are... This can be expressed as Formula 1:

[0055]

[0056] In Formula 1, l and h represent the schematic diagram of the electrode channel distribution. Figure 2 In this function, the lengths of the x and y axes are given, and d is the distance between the two electrodes. The weights are determined based on the relative and regional positions of the electrodes. Specifically, for electrode channels in the left (or right) brain region, the closer the two electrode channels are, the more similar they are considered, and the closer their data distributions are, resulting in a larger weight. This is to minimize the difference in data distribution between the two electrodes during network training. Conversely, the farther apart the two electrode channels are, the less similar they are considered, and the potentially different data distributions are, resulting in a smaller weight. During network training, the distance between the data distributions of the two electrodes is dynamically reduced based on the weights.

[0057] 2. Construct a loss function for the data distribution difference between electrodes based on electrode distance:

[0058] 1) Loss function for data distribution differences between user internal electrodes:

[0059] Loss function based on the distribution difference between source and target domain user intra-user electrode data in the electrode distance As shown in Formula 2:

[0060]

[0061] In formula 2, The weight function defined in Section 1, E represents the expectation associated with the source domain, target domain, and electrode. S E T e represents the expected values ​​of the source domain data and the target domain data. S ,e Tdenotes the number of electrode channels of source domain data and target domain data; φ[·] is a nonlinear mapping function that can map features to RKHS space; z S ,z T denotes the deep features of source domain data and target domain data extracted by the network, n S ,n T denotes the number of samples of source domain data and target domain data.

[0062] Thus, the data distribution difference loss function between intra-user electrodes based on electrode distance can be expressed as formula 3:

[0063] The data distribution difference loss function between source domain and target domain electrodes based on electrode distance.

[0064] The electrode data distribution difference loss function between source domain and target domain users based on electrode distance As shown in formula 4:

[0065]

[0066] 3. Construct the overall structure of the network:

[0067] The core part of the constructed network is in the full connection layer part. By minimizing each loss defined in sections 1-2 during network training, Figure 3 is a simplified network structure diagram.

[0068] For the data distribution difference loss function between intra-user electrodes In the network training process, minimize When the distance between the data of each electrode in the source domain (and the target domain) is as close as possible, the deep features output by the network of the source domain (and the target domain) will have a lower electrode channel difference; for the distribution difference loss function of the electrode data between the source domain and the target domain users Its role is to minimize the distance between the data of each electrode of the source domain user and each electrode of the target domain user during network training, and finally the deep features output by the network of the source domain and the target domain will have a lower electrode channel difference;

[0069] During network training, the loss function and are not minimized at the same time; Figure 2 is a simplified network structure diagram. Assuming that the training round number is 500 rounds, in the first 200 rounds of training, minimize the loss to narrow the distribution distance between the intra-source domain and intra-target domain electrode data For convenience of understanding, the specific format of the data is shown inFigure 2 It was also indicated in the text, for example The format is [B,F,C,S1], where B represents the batch size, F represents the number of filter channels, C represents the number of electrode channels, and S1 represents the number of sampling points per electrode channel; in the last 200 training rounds, the loss function is minimized. To minimize electrode data between the source and target domains The distribution distance between them; at the same time, the loss function With loss function There is a fully connected layer FC Layer 1 connecting them, which is a 3D fully connected layer; the last 100 rounds are used to zoom in on the overall data of the source and target domains. The distance, i.e., minimizing the loss D S-T Among them, features It is a feature It is obtained by stretching (Flatten) and passing it through a fully connected FC Layer 2.

[0070] As an example, we conducted a motor imagery (MI) EEG classification experiment using the BCI Competition IVIIa transfer learning dataset. The dataset contains motor imagery EEG data for 5 users “aa,al,av,aw,ay”, with 118 EEG data channels and 280 EEG data samples for each user.

[0071] Referring to the experimental procedures of existing motor imagery EEG domain adaptation experiments, the single-source domain user to single-target domain user (STS) experiment needs to be completed between every 2 users out of 5 users, that is, a total of 20 STS tasks.

[0072] The steps of a single STS task are as follows: The same number of samples are selected for both the original domain and the target domain as the training set, denoted as x respectively. S(n) x T(n) The validation set data comes from the original domain and is denoted as v. s(n) The ratio of training set to validation set samples is 8:2. During training, the accuracy achieved by the model on the validation set is recorded, and this accuracy is selected as the final result.

[0073] The average accuracy of the 20 STS tasks is the final accuracy of the model. In order to make full use of all samples and make the results convincing, the samples were randomly divided into ten parts for ten-fold cross-validation (CV). Different training and validation sets were selected in each fold for 20 STS tasks. Table 1 shows the accuracy of different transfer tasks in different folds in the ten-fold cross-validation.

[0074] Table 1 Ten-fold cross-validation classification results

[0075]

[0076] Table 2 compares the accuracy of the invented adaptive network (denoted as DEANN) and two existing state-of-the-art cross-subject motor imagery EEG classification models, namely DDC (Deep domain confusion) and DDAN (Deep domain adaptation network).

[0077] Table 2 Each network accuracy, total training time, average time statistics

[0078]

[0079] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art, according to the technical solution and the inventive concept of the present application, makes equivalent replacement or change within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. An adaptive algorithm for cross-subject motor imagery classification, characterized in that, Comprising the following steps: S1 : constructing an electrode distance-based inter-electrode loss weight function, the inter-electrode loss weight function is represented as: ； Wherein: X is the horizontal coordinate data of the electrode channel of the data set, Y is the vertical coordinate data of the electrode channel of the data set, l, h are the maximum values of the horizontal and vertical coordinates of the electrode channel of the data set respectively, and d is the distance between the two electrodes; S2: constructing an electrode distance-based inter-electrode data distribution difference loss function, the electrode distance-based inter-electrode data distribution difference loss function including the following two kinds: an electrode distance-based intra-user inter-electrode data distribution difference loss function , and an electrode distance-based source domain and target domain inter-electrode data distribution difference loss function ; the data distribution difference loss function satisfies: ； wherein: and are electrode-to-electrode loss weight functions, is an electrode distance-based source domain data distribution discrepancy loss function, is an electrode distance-based target domain intra-user electrode data distribution discrepancy loss function, and are determined by the following equations, respectively: wherein: is a loss weight function between electrodes in S1, and denote the source domain and electrode related expectation and the target domain and electrode related expectation, respectively, E S and E T denote the expectation of the source domain data and the target domain data, respectively, e S and e T denote the number of electrode channels of the source domain data and the target domain data, respectively, Φ[·] is a nonlinear mapping function that can map features to RKHS space, Z S and Z T denote the deep features of the source domain data and the target domain data extracted by the network, respectively, n S and n T denote the number of samples of the source domain data and the target domain data, respectively, the electrode data distribution difference loss function satisfies: S3: Constructing a network overall structure, minimizing the electrode distance loss weight function and the data distribution difference loss function through training, wherein the network overall structure comprises a feature extraction layer, a first full connection layer connected to the feature extraction layer, and a second full connection layer connected to the first full connection layer; By inputting the electrode distance loss weight function and the electrode distance data distribution difference loss function, and training through the network overall structure to minimize the two loss functions, the accurate classification of the cross-user motor imagery electroencephalogram data from the single-source domain user to the single-target domain user is completed.

2. The adaptive algorithm for cross-subject motor imagery classification according to claim 1, wherein, The data set is the "BCI Competition IV IIa" data set.

3. The adaptive algorithm for cross-subject motor imagery classification according to any of claims 1 to 2, characterized in that: In S3, the method for minimizing the electrode distance loss weight function and the data distribution difference loss function is a convolutional neural network training method.

4. The adaptive algorithm for cross-subject motor imagery classification according to claim 1, wherein: In S3, With The minimization order is: first minimize , then minimize .

Images