Electroencephalogram signal transfer learning method and device

Abstract

The application discloses an electroencephalogram signal transfer learning method and device, and relates to the technical field of emotional brain-computer interaction, and comprises the following steps: collecting electroencephalogram signals of different individuals; extracting electroencephalogram signal feature data of different individuals, and taking the electroencephalogram signal feature data of a target individual among the different individuals as target domain data; for each individual among the individuals except the target individual, an initial transfer network is respectively constructed; based on the initial transfer network, the electroencephalogram signal feature data of each individual among the other individuals is transferred to the target domain data, and initial transfer data corresponding to each individual is obtained; the data sequence similarity between the initial transfer data and the target domain data is calculated, and the initial transfer network is updated and iterated based on the data sequence similarity, until a preset condition is met, and the final transferred data of each individual is output; and a sentiment recognition model is trained according to the transferred data. The application can effectively eliminate the EEG data differences of different individuals.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of affective brain-computer interaction, in particular to an electroencephalogram signal transfer learning method and device. BACKGROUND

[0002] At present, the thought interaction based on BCI (Brain-Computer Interface) is widely researched in various industries. EEG (electroencephalogram) is a special physiological signal that can best reflect the cognitive activity of the human brain. The output of the signal is not subject to the subjective control of the subject, and directly reflects the real activity intention of the person, which is a key indicator of the cognitive state detection task. In the BCI system, the cognitive state detection method based on EEG is more and more concerned due to its convenient operation, non-invasive, low risk, low cost, strong mobility and flexibility of the device, and its application scenarios are very wide, including disease diagnosis, driver fatigue state detection and emotion recognition.

[0003] Due to the differences in behavior style, physiological feeling and body function of different individuals, the differences will affect the training accuracy of the emotion recognition model, and also limit the generalization ability of the emotion recognition model between subjects. An important premise of conventional machine learning is that the existing data and unknown data conform to the same rule, that is, conform to the same distribution. In the individual independent problem based on EEG data, the EEG data of different individuals is different in distribution, which leads to poor effect of the conventional method. Therefore, the prior art proposes to introduce a transfer learning method to narrow the distribution of EEG data of different individuals.

[0004] At present, in the existing cross-subject transfer learning method, the EEG data of different individuals usually shares a transfer model. However, since the shared transfer model has poor generalization performance for different individuals, it is difficult to eliminate the differences in EEG data of different individuals, which leads to limited effect of narrowing the distribution of data of different individuals, thereby affecting the recognition accuracy of the finally trained emotion recognition model, and further leading to hindering the practical application and popularization of brain-computer interaction. SUMMARY

[0005] The present application provides an electroencephalogram signal transfer learning method and device, which can effectively eliminate the differences in EEG data of different individuals, thereby improving the recognition accuracy of the emotion recognition model.

[0006] According to a first aspect of the embodiment of the present application, an electroencephalogram signal transfer learning method is provided, comprising:

[0007] Collecting the whole brain area of different individuals to obtain the corresponding EEG signals of different individuals;

[0008] extract brain electrical signal feature data of the different individuals based on the brain electrical signals, and take brain electrical signal feature data of a target individual among the different individuals as target domain data;

[0009] construct an initial migration network for each individual among the other individuals except the target individual respectively;

[0010] migrate brain electrical signal feature data of each individual among the other individuals to the target domain data based on the initial migration network, to obtain initial migration data corresponding to the each individual;

[0011] calculate data sequence similarity between the initial migration data and the target domain data, and update the initial migration network of the each individual based on the data sequence similarity until a preset condition is met, and output data after final migration of the each individual;

[0012] train a sentiment recognition model according to the data after final migration of the each individual.

[0013] According to a second aspect of the embodiment of the present application, a brain electrical signal migration learning device is provided, comprising:

[0014] a collection unit configured to collect different individuals in whole brain regions to obtain brain electrical signals corresponding to the different individuals;

[0015] an extraction unit configured to extract brain electrical signal feature data of the different individuals based on the brain electrical signals, and take brain electrical signal feature data of a target individual among the different individuals as target domain data;

[0016] a construction unit configured to construct an initial migration network for each individual among the other individuals except the target individual respectively;

[0017] a migration unit configured to migrate brain electrical signal feature data of each individual among the other individuals to the target domain data based on the initial migration network, to obtain initial migration data corresponding to the each individual;

[0018] an iteration unit configured to calculate data sequence similarity between the initial migration data and the target domain data, and update the initial migration network of the each individual based on the data sequence similarity until a preset condition is met, and output data after final migration of the each individual;

[0019] a training unit configured to train a sentiment recognition model according to the data after final migration of the each individual.

[0020] According to a third aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the following steps:

[0021] The whole brain region of different individuals is collected to obtain the electroencephalogram signals corresponding to the different individuals;

[0022] Based on the electroencephalogram signals, the electroencephalogram signal feature data of the different individuals is extracted, and the electroencephalogram signal feature data of a target individual in the different individuals is taken as target domain data;

[0023] For each individual other than the target individual, an initial migration network is constructed respectively;

[0024] Based on the initial migration network, the electroencephalogram signal feature data of each individual in the other individuals is migrated to the target domain data to obtain initial migration data corresponding to the each individual;

[0025] The data sequence similarity between the initial migration data and the target domain data is calculated, and the initial migration network of the each individual is updated iteratively based on the data sequence similarity, until a preset condition is met, and the final migrated data of the each individual is output;

[0026] A sentiment recognition model is trained according to the final migrated data of the each individual.

[0027] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program:

[0028] The whole brain region of different individuals is collected to obtain the electroencephalogram signals corresponding to the different individuals;

[0029] Based on the electroencephalogram signals, the electroencephalogram signal feature data of the different individuals is extracted, and the electroencephalogram signal feature data of a target individual in the different individuals is taken as target domain data;

[0030] For each individual other than the target individual, an initial migration network is constructed respectively;

[0031] Based on the initial migration network, the electroencephalogram signal feature data of each individual in the other individuals is migrated to the target domain data to obtain initial migration data corresponding to the each individual;

[0032] Calculate the data sequence similarity between the initial migration data and the target domain data, and perform an update iteration on the initial migration network of each individual based on the data sequence similarity until a preset condition is met, and output the data of each individual after final migration.

[0033] According to the data of each individual after final migration, a sentiment recognition model is trained.

[0034] The innovation points of the embodiments of the present application include:

[0035] 1. For different individuals, a migration network is trained separately to eliminate the differences in EEG data of different individuals and improve the recognition accuracy of the sentiment recognition model.

[0036] 2. A unique domain linear mapping network is constructed, and the migration network of a simple linear model greatly reduces the parameter data amount of linear mapping, makes up for the defect that the overall data calculation amount is large due to the large model size of the ordinary mapping network model, overcomes the technical prejudice that the cross-individual domain mapping is not suitable for EEG data migration learning, thereby greatly reducing the batch training time, and further reducing the difficulty and cost of batch training.

[0037] 3. Batch normalization processing saves data processing time, and batch reduces the differences in electroencephalogram data of all collected individuals.

[0038] The EEG signal migration learning method and device provided by the application can collect different individuals in the whole brain area, obtain the EEG signals corresponding to different individuals, then extract the EEG signal feature data of different individuals based on the EEG signals, take the EEG signal feature data of a target individual in the different individuals as target domain data, then construct an initial migration network for each individual other than the target individual, migrate the EEG signal feature data of each individual other than the target individual to the target domain data based on the initial migration network, obtain the initial migration data corresponding to each individual, further calculate the data sequence similarity between the initial migration data and the target domain data, update and iterate the initial migration network of each individual based on the data sequence similarity, output the data of each individual after final migration when a preset condition is met, and finally train a sentiment recognition model according to the data of each individual after final migration.

[0039] The above description is only a summary of the technical solutions of the application. In order to more clearly understand the technical means of the application, the application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0041] Figure 1A flowchart of an electroencephalogram signal transfer learning method provided by an embodiment of the present application is shown.

[0042] Figure 2 A structural diagram of an electroencephalogram signal transfer learning device provided by an embodiment of the present application is shown.

[0043] Figure 3 A structural diagram of another fuel cell fault diagnosis model construction device provided by an embodiment of the present application is shown.

[0044] Figure 4 A structural diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the scope of protection of the present application.

[0046] It should be noted that the terms “include” and “have” and any variations thereof in the embodiments of the present application and the drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed or optionally further includes other steps or units inherent to the process, method, product or device.

[0047] The transfer model commonly used in the prior art has poor generalization performance for different individuals, so it is difficult to eliminate the differences in EEG data of different individuals, thereby affecting the recognition accuracy of the finally trained emotion recognition model.

[0048] In order to overcome the above defects, an electroencephalogram signal transfer learning method is provided in the embodiments of the present application, as shown in the figure, which comprises the following steps. Figure 1

[0049] Step 101: Collecting the whole brain area of different individuals to obtain the electroencephalogram signals corresponding to different individuals.

[0050] The different individuals can be specifically human, and the number of individuals collected is at least two.

[0051] The embodiments of the present application are mainly applicable to the scene of data transfer of electroencephalogram signals of different individuals in emotional brain-computer interaction, and training of an emotion recognition model using the transferred data. The execution subject of the embodiments of the present application is a device or equipment capable of data transfer and training of an emotion recognition model.​

[0052] For the embodiment of the present application, the electroencephalogram signal data of different individuals can be collected by a 62-channel collection method, the sampling frequency is 200HZ, and the sample length is 60-300 seconds. In addition, if no data collection is performed, the electroencephalogram signals of different individuals can also be obtained from the existing SEED data set.

[0053] Further, the collected electroencephalogram signals of different individuals are preprocessed. For the preprocessing process, the method comprises: based on the electroencephalogram signals, bandwidth filtering is performed to obtain filtered electroencephalogram signals; the filtered electroencephalogram signals are respectively subjected to waveform artifact removal and time window division to obtain preprocessed electroencephalogram signals. The preprocessed electroencephalogram signals have multiple channels, multiple frequency bands and unified time units.

[0054] Specifically, first, the electroencephalogram signals of different individuals are subjected to bandwidth filtering, and the corresponding frequency band frequency range information is output. The frequency band can be 5 frequency bands, but is not limited thereto. Then, the filtered electroencephalogram signals are subjected to principal component analysis, i.e., the waveform artifacts in the electroencephalogram signals are removed, and the waveform artifacts mainly include eye artifacts and electromyogram signals. At the same time, time window division is performed to obtain the minimum time unit for electroencephalogram signal analysis. In this way, the electroencephalogram information with multiple channels, multiple frequency bands and unified time units, i.e., the preprocessed electroencephalogram signals, can be obtained.

[0055] Step 102, based on the electroencephalogram signals, the electroencephalogram signal feature data of the different individuals is extracted, and the electroencephalogram signal feature data of a target individual in the different individuals is taken as target domain data.

[0056] The target individual can be any individual in the different individuals.

[0057] For the embodiment of the present application, after the electroencephalogram signals are preprocessed, the electroencephalogram signal feature data of different individuals can be extracted based on the preprocessed electroencephalogram signals. For the extraction process of the feature data, the method comprises: calculating the autocorrelation function corresponding to the preprocessed electroencephalogram signals; performing Fourier transform on the autocorrelation function to obtain the power spectral density of the preprocessed electroencephalogram signals; and determining the power spectral density as the electroencephalogram signal feature data of the different individuals.

[0058] It should be noted that in the embodiment of the present application, the extraction method of the power spectral density feature is not limited to the autocorrelation function method, and other methods can also be used, which are not limited in the embodiment of the present application.

[0059] Further, in order to reduce the data calculation amount and improve the training effect of the model, the embodiment of the present application can also perform channel screening and frequency band screening on the brain electrical signal feature data of different individuals. When performing channel screening, the overall distribution of the brain electrical signal feature data of two different channels can be determined by a non-parametric significance test method whether there is a significant difference, if there is no significant difference, only the brain electrical signal feature data of one of the channels needs to be retained. Similarly, when performing frequency band screening, the overall distribution of the brain electrical signal feature data of two different frequency bands can be determined by a non-parametric significance test method whether there is a significant difference, if there is no significant difference, only the brain electrical signal feature data of one of the frequency bands needs to be retained. Thus, according to the above manner, the screened brain electrical signal feature data corresponding to different individuals can be determined through frequency band screening and channel screening.

[0060] It should be noted that the non-parametric significance test method used in the embodiment of the present application can be Mann-Whitney U test, K-S test and the like.

[0061] Further, the screened brain electrical signal feature data corresponding to different individuals is added to the feature data set, at the same time, a random individual is selected from different individuals as a target individual, and the screened brain electrical signal feature data of the target individual in the feature data set is taken as target domain data, and the screened brain electrical signal feature data of all other individuals is taken as source domain data. Further, the target domain data is further divided into target domain target data and target domain test data.

[0062] Specifically, when selecting the target domain target data, the brain electrical signal feature data of the target individual under different frequency bands, different channels and different emotional labels can be extracted from the target domain data to be one of them to be taken as the target domain target data, and the remaining data of the target individual in the target domain data is taken as the target domain test data, so as to perform data migration and emotion recognition model test based on the target domain target data and the target domain test data.

[0063] Further, batch normalization processing also needs to be performed on the brain electrical signal feature data in the feature data set, and the specific formula of the normalization processing is as follows:

[0064]

[0065] Wherein, X k is the brain electrical signal feature data in a single time window, is the normalized brain electrical signal feature data, X max and X min are the maximum value and the minimum value of the feature data in the continuous time period where the time window is located.

[0066] Step 103: For each individual other than the target individual, construct an initial migration network.

[0067] Specifically, the initial transfer network can be an initial depth convolutional layer, and the initial depth convolutional layer is different for different individuals.

[0068] To effectively eliminate the differences in EEG data among different individuals, this embodiment of the invention constructs an initial transfer network, i.e., an initial deep convolutional layer, for each of the other individuals. By iteratively training the parameters of this initial deep convolutional layer, a trained deep convolutional layer and the transferred data for each of the other individuals can be obtained. The specific formula for the linear transformation based on the deep convolutional layer is as follows:

[0069] X DCNN =W DCNN X ELBN +B DCNN

[0070] Among them, W DCNN and B DCNN These are the scaling and translation parameters, i.e., the weights and biases of the depthwise convolutional layer. Initial values ​​for the weights and biases are given in the initial depthwise convolutional layer, X. ELBN For each of the other individuals, X represents the normalized EEG signal feature data. DCNN This is the data after being transferred through a deep convolutional layer.

[0071] Furthermore, the above formula and the batch normalization linear calculation process can be combined into a single linear calculation formula, as follows:

[0072] X DCNN =W DCNN (w ELBN {X k}+b ELBN )+B DCNN =W{X k}+B

[0073] Therefore, each sequence segment in each frequency band under each channel can be independently computed and linearly transferred. Based on this, the scaling parameter W and translation parameter B in the deep convolutional layer can be used as channel and frequency band feature values ​​of the EEG sequence.

[0074] Step 104: Based on the initial transfer network, transfer the EEG signal feature data of each of the other individuals to the target domain data to obtain the initial transfer data corresponding to each individual.

[0075] For the embodiment of the application, when training the deep convolutional layer, an initial value can be given to the weight term and the bias term respectively, that is, an initial deep convolutional layer is constructed, and then the electroencephalogram feature data of each individual in the other individuals is migrated to the target domain target data based on the initial deep convolutional layer to obtain the initial migration data corresponding to each individual.

[0076] When performing data migration, the channels, frequency bands and emotional labels are one-to-one corresponding data migration. Based on this, the method comprises: using the initial deep convolutional layer to migrate the electroencephalogram feature data of each individual in the other individuals to the electroencephalogram feature data in the target domain target data which has the same channel, frequency band and emotional label as it, to obtain the initial migration data corresponding to each individual. Wherein, the emotional label includes happy, sad and neutral.

[0077] For example, the electroencephalogram feature data of the other individuals "happy" is migrated to the electroencephalogram feature data of the target individual "happy" in the target domain target data, and the electroencephalogram feature data of the other individuals "sad" is migrated to the electroencephalogram feature data of the target individual "sad" in the target domain target data.

[0078] Therefore, the deep convolutional layer can realize separate calculation of different electroencephalogram channel data, separate calculation of different frequency band data, and separate calculation of different emotional label data. On this basis, the scaling parameter W and the translation parameter B in the deep convolutional layer can represent the migration path, and when the data of the other individuals is migrated to the same target individual data, the difference between the migration paths can reflect the difference between the migrated data samples.

[0079] Step 105, calculate the data sequence similarity between the initial migration data and the target domain data, and update and iterate the initial migration network of each individual based on the data sequence similarity until the preset condition is met, and output the final migrated data of each individual.

[0080] Wherein, the initial migration data and the target domain target data in the target domain data are time-based data sequences.

[0081] For the embodiment of the application, in order to continuously iterate and train the initial migration network (initial deep convolutional layer), it is necessary to calculate the data sequence similarity between the initial migration data and the target domain target data, and update and iterate the initial deep convolutional layer of each individual based on the data sequence similarity until the preset condition is met, and output the final migrated data of each individual. Wherein, the preset condition can be that the preset iteration number is reached or the loss function value reaches the preset threshold. For example, the preset iteration number is set to 150 times, and the preset threshold of the loss function value is within 100.

[0082] Further, in the calculation of the data sequence similarity, the method specifically comprises: calculating a distance matrix between each element according to the initial migration data and the electroencephalogram signal feature data in the target domain target data having the same channel, frequency band and emotion label as the initial migration data; finding a target path from the top left corner to the bottom right corner of the distance matrix, so that the sum of the elements on the target path is minimum; determining the sum of the elements on the target path as the data sequence similarity.

[0083] Specifically, in the iterative training of the migration network (deep convolutional layer), the embodiment of the application constructs a loss function based on the data sequence similarity between the source domain data and the target domain target data. Since the similarity between two data sequences is usually represented by dynamic time warping distance (DTW distance), the loss function is also called DTW distance loss function. The specific calculation formula of the DTW distance loss function is as follows:

[0084] X = {x1, x2, …, x m}

[0085] Y = {y1, y2, …, y n}

[0086] d(i, j) = (x i -y j ) 2

[0087] D(i, h) = d(i, h) + min{D(i-1, h), D(i, j-1), D(i-1, j-1)}

[0088] Δ(X, Y) = D(m, n)

[0089]

[0090] Wherein, X represents the migrated data, Y represents the target domain target data, X and Y are data sequences, and X sequence includes scaling parameter W and translation parameter B, D(i, j) is a distance matrix, Δ(X, Y) is the sum of the elements on the path from the top left corner to the bottom right corner of the distance matrix, i.e. the distance value, DTW(X, Y) is the DTW distance value, which can be used to represent the sequence similarity between X and Y, the DTW distance value is essentially the minimum sum of the elements on the path from the top left corner to the bottom right corner of the distance matrix, and A is a boundary condition.

[0091] Further, after the loss function is constructed, the obtained initial migration data can be brought into the loss function, so that the data sequence similarity between the initial migration data and the target data of the target domain, i.e., the loss function value, can be calculated, and then the parameters in the initial migration network of the other individual are iteratively updated based on the loss function value until a preset iteration number is reached or the loss function value reaches a preset threshold, and the trained migration network of the other individual and the data of each individual in the other individual after final migration are output.

[0092] It should be noted that, in order to ensure the accuracy of the migration network training, when calculating the loss function value, i.e., the data sequence similarity, the source domain data and the target data of the target domain need to be one-to-one corresponding according to the channel, frequency band and emotion label, i.e., for any one individual data in the source domain data, the data with the same channel, frequency band and emotion label in the target data of the target domain is found first, and then the two are calculated for sequence similarity, and then the similarity calculation results corresponding to each data of the individual are added to obtain the final loss function value.

[0093] Further, in order to be able to iteratively based on the constructed loss function, the DTW distance loss function also needs to be differentiable, and the specific formula is as follows:

[0094]

[0095]

[0096] Wherein, min γ {a1, …, a1} is a generalized minimum operator, a is a calculated vector, and γ is a smoothing parameter, Y is 0.1 in the embodiment of the application.

[0097] According to the above manner, the data of different individuals after migration can be obtained by iteratively training the migration network of different individuals.

[0098] Step 106, training a sentiment recognition model according to the data of each individual after final migration.

[0099] The trained sentiment recognition model can be KNN, LR, SVM and CNN model.

[0100] For the embodiment of the application, the data of each individual after final migration can be used as training data, and the sentiment recognition model can be trained based on the training data. Further, the sentiment recognition model can be tested based on the target domain test data to obtain the test accuracy of the sentiment recognition model.

[0101] In addition, the embodiment of the present application can also construct a sentiment recognition model based on the migrated data and the pre-migration data respectively, and then test the effects of the two models by using the target domain test data, so as to find that the sentiment recognition model trained by the migrated data has better recognition effect.

[0102] The EEG signal transfer learning method provided by the embodiment of the present application can train a transfer network for different individuals separately, and transfer data of other individuals to target individual data by using the transfer network. Since the above-mentioned model framework can be used on different individuals and good effects can be achieved, the model framework has good generalization performance, thereby effectively eliminating the differences between EEG data of different individuals and improving the recognition accuracy of the sentiment recognition model, so that each individual can obtain a high-precision model for individual differences. Further, since the embodiment of the present application constructs a unique domain linear mapping network, i.e., a transfer network of a simple linear model, for different individuals, the parameter data amount of linear mapping is greatly reduced, the defect of large overall data calculation amount caused by large model size of an ordinary mapping network model is compensated, and the batch training time is greatly reduced, the difficulty and cost of batch training are reduced, which is conducive to promoting the practical application of brain-computer interaction.

[0103] Further, as a specific implementation of Figure 1 , the embodiment of the present application provides an EEG signal transfer learning device, as shown in Figure 2 , the device comprises an acquisition unit 31, an extraction unit 32, a construction unit 33, a transfer unit 34, an iteration unit 35 and a training unit 36.

[0104] The acquisition unit 31 can be used for whole brain area acquisition of different individuals to obtain EEG signals corresponding to different individuals.

[0105] The extraction unit 32 can be used for extracting EEG signal feature data of different individuals based on the EEG signals, and taking EEG signal feature data of a target individual in the different individuals as target domain data.

[0106] The construction unit 33 can be used for constructing an initial transfer network for each individual of the other individuals except the target individual.

[0107] The transfer unit 34 can be used for transferring EEG signal feature data of each individual of the other individuals to the target domain data based on the initial transfer network to obtain initial transfer data corresponding to each individual.

[0108] The iteration unit 35 can be configured to calculate a data sequence similarity between the initial migration data and the target domain data, and perform update iteration on the initial migration network of each individual based on the data sequence similarity until a preset condition is met, and output final migration data of each individual.

[0109] The training unit 36 can be configured to train a sentiment recognition model according to the final migration data of each individual.

[0110] In a specific application scenario, as shown in Figure 3 The device further includes a preprocessing unit 37.

[0111] The preprocessing unit 37 can be configured to perform bandwidth filtering based on the electroencephalogram signal to obtain a filtered electroencephalogram signal, and perform waveform artifact removal and time window division on the filtered electroencephalogram signal respectively to obtain a preprocessed electroencephalogram signal, wherein the preprocessed electroencephalogram signal has multiple channels, multiple frequency bands, and a unified time unit.

[0112] The extraction unit 32 can be specifically configured to extract the electroencephalogram signal feature data of different individuals based on the preprocessed electroencephalogram signal.

[0113] Further, the extraction unit 32 can be specifically configured to calculate an autocorrelation function corresponding to the preprocessed electroencephalogram signal, perform Fourier transform on the autocorrelation function to obtain a power spectral density of the preprocessed electroencephalogram signal, and determine the power spectral density as the electroencephalogram signal feature data of different individuals.

[0114] In a specific application scenario, the device further includes a screening unit 38.

[0115] The screening unit 38 can be configured to perform channel screening and frequency band screening on the computer signal feature data of different individuals respectively to obtain screened computer signal feature data corresponding to different individuals.

[0116] In a specific application scenario, the migration unit 34 includes a division module 341 and a migration module 342.

[0117] The division module 341 can be configured to divide the target domain data into target domain target data and target domain test data.

[0118] The migration module 342 can be configured to migrate the electroencephalogram signal feature data of each individual in the other individuals to the target domain target data based on the initial deep convolutional layer to obtain initial migration data corresponding to each individual.

[0119] The iteration unit 35 can be specifically configured to calculate a data sequence similarity between the initial migration data and the target domain target data, and update and iterate the initial deep convolutional layer of each individual based on the data sequence similarity until a preset condition is met, and output the final migration data of each individual.

[0120] Further, the migration module 342 can be specifically configured to use the initial deep convolutional layer to migrate the electroencephalogram signal feature data of each individual in the other individuals to the electroencephalogram signal feature data in the target domain target data which has the same channel, frequency band and emotion label as the electroencephalogram signal feature data, to obtain the initial migration data corresponding to each individual.

[0121] Further, the iteration unit 35 can be specifically configured to calculate a distance matrix between elements according to the initial migration data and the electroencephalogram signal feature data in the target domain target data which has the same channel, frequency band and emotion label as the initial migration data; find a target path from the top left corner to the bottom right corner of the distance matrix, so that the sum of elements on the target path is minimum; and determine the sum of elements on the target path as the data sequence similarity.

[0122] In a specific application scenario, the device further comprises a test unit 39.

[0123] The test unit 39 can be configured to test the emotion recognition model based on the target domain test data, to obtain a test precision corresponding to the emotion recognition model.

[0124] It should be noted that other corresponding descriptions of the functions of the electroencephalogram signal migration learning device provided by the embodiments of the present application can be referred to the corresponding descriptions of the method shown in Figure 1 The method shown in the foregoing description will not be repeated here.

[0125] Based on the above, the method shown in the foregoing description will not be repeated here. Figure 1Accordingly, this embodiment of the invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps: collecting whole-brain region data from different individuals to obtain corresponding EEG signals; extracting EEG signal feature data from the different individuals based on the EEG signals, and using the EEG signal feature data of a target individual as target domain data; constructing an initial transfer network for each individual other than the target individual; transferring the EEG signal feature data of each individual to the target domain data based on the initial transfer network to obtain initial transfer data for each individual; calculating the data sequence similarity between the initial transfer data and the target domain data, and updating and iterating the initial transfer network for each individual based on the data sequence similarity until a preset condition is met, and outputting the final transferred data for each individual; and training an emotion recognition model based on the final transferred data for each individual.

[0126] Based on the above, Figure 1 The method shown and as Figure 2 The embodiment of the device shown in the invention also provides a physical structural diagram of an electronic device, such as... Figure 4 As shown, the electronic device includes: a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor. Both the memory 42 and the processor 41 are mounted on a bus 43. When the processor 41 executes the program, it performs the following steps: whole-brain region acquisition of different individuals to obtain corresponding EEG signals; extraction of EEG signal feature data from the different individuals based on the EEG signals, and using the EEG signal feature data of a target individual as target domain data; construction of an initial transfer network for each individual other than the target individual; transfer of the EEG signal feature data of each individual to the target domain data based on the initial transfer network, obtaining initial transfer data for each individual; calculation of the data sequence similarity between the initial transfer data and the target domain data, and updating and iterating the initial transfer network for each individual based on the data sequence similarity until a preset condition is met, outputting the final transferred data for each individual; and training an emotion recognition model based on the final transferred data for each individual.

[0127] The embodiment of the present application trains the transfer network for different individuals, and transfers the data of other individuals to the data of the target individual by using the transfer network. Since the above model framework can be used on different individuals and good results can be achieved, the model framework has good generalization performance, thereby effectively eliminating the EEG data differences of different individuals and improving the recognition accuracy of the emotion recognition model. That is, each individual can obtain a high-precision model for the individual difference mechanism of the individual. Further, since the embodiment of the present application constructs a unique domain linear mapping network, i.e., a transfer network of a simple linear model, for different individuals, the parameter data amount of linear mapping is greatly reduced, the defect of large overall data calculation amount caused by the large model size of the ordinary mapping network model is compensated, thereby the batch training time can be greatly reduced, the difficulty and cost of batch training are reduced, and the practical application of brain-computer interaction is promoted.

[0128] Those skilled in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or flows in the drawings are not necessarily necessary for implementing the present application.

[0129] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the embodiment description, or can be changed and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further split into multiple sub-modules.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An electroencephalogram signal transfer learning method, characterized in that, The method comprises the following steps: Collecting the whole brain area of different individuals to obtain the corresponding electroencephalogram signals of different individuals; Based on the electroencephalogram signals, the electroencephalogram signal feature data of the different individuals is extracted, and the electroencephalogram signal feature data of a target individual in the different individuals is taken as target domain data; For each individual in the individuals other than the target individual, an initial migration network is constructed respectively; Based on the initial migration network, the electroencephalogram signal feature data of each individual in the other individuals is migrated to the target domain data to obtain the initial migration data corresponding to each individual; The data sequence similarity between the initial migration data and the target domain data is calculated, and the initial migration network of each individual is updated iteratively based on the data sequence similarity until a preset condition is met, and the final migrated data of each individual is output; According to the final migrated data of each individual, a sentiment recognition model is trained; The initial migration network is an initial deep convolutional layer, and the migration of the electroencephalogram signal feature data of each individual in the other individuals to the target domain data based on the initial migration network to obtain the initial migration data corresponding to each individual comprises: The target domain data is divided into target domain target data and target domain test data; Based on the initial deep convolutional layer, the electroencephalogram signal feature data of each individual in the other individuals is migrated to the target domain target data to obtain the initial migration data corresponding to each individual; The migration of the electroencephalogram signal feature data of each individual in the other individuals to the target domain target data based on the initial deep convolutional layer to obtain the initial migration data corresponding to each individual comprises: Using the initial deep convolutional layer, the electroencephalogram signal feature data of each individual in the other individuals is migrated to the electroencephalogram signal feature data in the target domain target data which has the same channel, frequency band and emotion label as the electroencephalogram signal feature data to obtain the initial migration data corresponding to each individual; The calculation of the data sequence similarity between the initial migration data and the target domain data, and the update iteration of the initial deep convolutional layer of each individual based on the data sequence similarity until a preset condition is met, and the final migrated data of each individual is output, comprises: The data sequence similarity between the initial migration data and the target domain target data is calculated, and the initial deep convolutional layer of each individual is updated iteratively based on the data sequence similarity until a preset condition is met, and the final migrated data of each individual is output; The calculation of the data sequence similarity between the initial migration data and the target domain target data comprises: According to the initial migration data, and the electroencephalogram signal feature data in the target domain target data which has the same channel, frequency band and emotion label as the initial migration data Calculate the distance matrix between elements; Find a target path from the top left corner to the bottom right corner of the distance matrix, so that the sum of the elements on the target path is minimum; determine a similarity of the data sequence to elements on the target path; and / or The method further includes: Testing the emotion recognition model based on the target domain test data to obtain a test accuracy corresponding to the emotion recognition model.

2. The method of claim 1, wherein, Before the extracting of the EEG signal feature data of the different individuals based on the EEG signals, the method further includes: Performing bandwidth filtering based on the EEG signals to obtain filtered EEG signals; Respectively performing waveform artifact removal and time window division on the filtered EEG signals to obtain preprocessed EEG signals, wherein the preprocessed EEG signals have multiple channels, multiple frequency bands, and a unified time unit; The extracting of the EEG signal feature data of the different individuals based on the EEG signals includes: Extracting the EEG signal feature data of the different individuals based on the preprocessed EEG signals.

3. The method of claim 2, wherein, The extracting of the EEG signal feature data of the different individuals based on the preprocessed EEG signals includes: Calculating an autocorrelation function corresponding to the preprocessed EEG signals; Performing Fourier transform on the autocorrelation function to obtain a power spectral density of the preprocessed EEG signals; Determining the power spectral density as the EEG signal feature data of the different individuals.

4. The method of claim 1, wherein, After the extracting of the EEG signal feature data of the different individuals based on the EEG signals, the method further includes: Respectively performing channel screening and frequency band screening on the EEG signal feature data of the different individuals to obtain screened EEG signal feature data corresponding to the different individuals; The determining of the EEG signal feature data of a target individual in the different individuals as target domain data includes: Determining the screened EEG signal feature data of the target individual in the different individuals as the target domain data.

5. An electroencephalogram signal transfer learning device, characterized by, The method includes: An acquisition unit configured to perform whole-brain region acquisition on different individuals to obtain EEG signals corresponding to the different individuals; An extraction unit configured to extract EEG signal feature data of the different individuals based on the EEG signals, and determine EEG signal feature data of a target individual in the different individuals as target domain data; An establishment unit configured to establish an initial migration network for each individual other than the target individual. a migration unit, configured to migrate, based on the initial migration network, the electroencephalogram feature data of each of the other individuals to the target domain data to obtain initial migration data corresponding to each of the individuals; wherein the initial migration network is an initial deep convolutional layer, and the migration unit is specifically configured to divide the target domain data into target domain target data and target domain test data; migrate, based on the initial deep convolutional layer, the electroencephalogram feature data of each of the other individuals to the target domain target data to obtain the initial migration data corresponding to each of the individuals; and further specifically configured to migrate, by using the initial deep convolutional layer, the electroencephalogram feature data of each of the other individuals to electroencephalogram feature data with the same channel, frequency band and emotional label in the target domain target data to obtain the initial migration data corresponding to each of the individuals; an iteration unit, configured to calculate a data sequence similarity between the initial migration data and the target domain data, and perform update iteration on the initial migration network of each of the individuals based on the data sequence similarity, until a preset condition is met, and output data after final migration of each of the individuals; the iteration unit is specifically configured to calculate a data sequence similarity between the initial migration data and the target domain target data, and perform update iteration on the initial deep convolutional layer of each of the individuals based on the data sequence similarity, until a preset condition is met, and output data after final migration of each of the individuals; the iteration unit is further specifically configured to calculate a distance matrix between elements according to the initial migration data and electroencephalogram feature data with the same channel, frequency band and emotional label in the target domain target data; find a target path from the top left corner to the bottom right corner of the distance matrix, so that the sum of elements on the target path is minimum; determine the sum of elements on the target path as the data sequence similarity; and / or, the iteration unit is further specifically configured to test the emotion recognition model based on the target domain test data to obtain a test precision corresponding to the emotion recognition model; a training unit, configured to train an emotion recognition model according to the data after final migration of each of the individuals.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1 to 4.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1 to 4.

Images