Electroencephalogram signal classification model training method, intent recognition method, device and medium

Abstract

The application provides a brain electrical signal classification model training method, an intention recognition method, equipment and a medium, and belongs to the technical field of machine learning. The method comprises the following steps: obtaining target domain characteristics of target domain data and source domain characteristics of source domain data; determining global time domain dependence of the target domain characteristics based on an attention mechanism; migrating the characteristics of the global time domain dependence to the source domain characteristics to obtain migrated source domain characteristics; and training an initial model based on the target domain characteristics and the migrated source domain characteristics to obtain a trained classification model. In the embodiment of the application, the initial model can be trained by using the brain electrical signal data of the first object and the brain electrical signal data of the target object, so that the demand for the brain electrical signal data of the target object is reduced, the training efficiency of the classification model is improved, the preparation time before use of the brain-computer interface system is reduced, and the trained classification model has better versatility.

Description

Technical Field

[0001] This application belongs to the field of machine learning technology, and in particular relates to a method for training an EEG signal classification model, an intention recognition method, a device, and a medium. Background Technology

[0002] Brain-computer interfaces (BCIs) refer to the connection created between the brain of a person or animal and an external device, enabling information exchange between the brain and the device. BCI-based systems can help users control devices to perform tasks using their brain signals. BCI systems have been widely applied in various fields. For example, in the gaming industry, BCI systems can control virtual characters on the game screen to perform corresponding actions, such as walking, jumping, and turning, based on the user's brainwave signals. After collecting the user's brainwave signals, determining the type of these signals to accurately analyze the corresponding control intentions is a crucial issue for BCI systems.

[0003] Currently, numerous machine learning methods are applied to the classification and analysis of electroencephalogram (EEG) signals. However, due to the significant differences in EEG signals among different users, classification models trained on data from a single user are difficult to apply to classifying the EEG signals of other users. This results in poor versatility of classification models in practical applications, making them unsuitable as universal classification models for multiple users, and also leading to low training efficiency. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method for training an EEG signal classification model, an intention recognition method, an apparatus, and a medium, which can solve the problem of low training efficiency of classification models.

[0005] The first aspect of this application provides a method for training an electroencephalogram (EEG) signal classification model, including:

[0006] The target domain features of the target domain data and the source domain features of the source domain data are obtained. The target domain data includes the EEG signal of the target object, and the source domain data includes the EEG signal of the first object, which includes objects other than the target object.

[0007] Based on the attention mechanism, the global temporal dependency of the target domain features is determined. The global temporal dependency is the correlation data between the feature data at various positions in the time domain.

[0008] The features that are dependent on the global time domain are transferred to the source domain features to obtain the transferred source domain features.

[0009] Based on the target domain features and the transferred source domain features, the initial model is trained to obtain the trained classification model.

[0010] In a first possible implementation of the first aspect, the global temporal dependency of the target domain features is determined based on an attention mechanism, including:

[0011] The feature channels of the target domain feature are converted into one dimension to obtain the first one-dimensional data.

[0012] Perform a linear transformation on the first one-dimensional data to obtain the first query and the first key.

[0013] Calculate the dot product between the first query and the transposed first key to obtain the global relevance matrix of the target domain features.

[0014] The global correlation matrix is ​​scaled and normalized to obtain the correlation weight matrix of the target domain features. The correlation weight matrix is ​​the global time domain dependency.

[0015] Based on the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the globally time-domain dependent features are transferred to the source domain features to obtain the transferred source domain features, including:

[0016] The feature channels of the source domain features are converted into one dimension to obtain the second one-dimensional data.

[0017] A linear transformation is performed on the second one-dimensional data to obtain the first value.

[0018] Calculate the dot product of the first value and the correlation weight matrix to obtain the transferred source domain features.

[0019] Based on the first possible implementation of the first aspect, in the third possible implementation of the first aspect, the feature channels of the target domain features are converted into one dimension to obtain the first one-dimensional data, including:

[0020] The target domain features are divided into multiple first slice features along the time dimension.

[0021] Each feature channel of the first slice is converted into one dimension to obtain multiple third one-dimensional data. The first one-dimensional data contains multiple third one-dimensional data.

[0022] Based on the second possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the feature channels of the source domain features are converted into one dimension to obtain second one-dimensional data, including:

[0023] The source domain features are divided into multiple second slice features along the time dimension.

[0024] Each feature channel of the second slice is converted into one dimension to obtain multiple fourth one-dimensional data. The second one-dimensional data contains multiple fourth one-dimensional data.

[0025] In the fifth possible implementation of the first aspect, obtaining the target domain features of the target domain data and the source domain features of the source domain data includes:

[0026] Perform temporal convolution and spatial convolution on the first data respectively to obtain the first temporal feature and the first spatial feature; the first data is either target domain data or source domain data.

[0027] Adjust the dimensions of the first time-domain feature and the first spatial-domain feature to be consistent.

[0028] The first feature is obtained by performing calculations based on the first time-domain feature and the first spatial-domain feature with consistent size; the first feature is either a target domain feature or a source domain feature.

[0029] In the sixth possible implementation of the first aspect, the initial model is trained based on the target domain features and the transferred source domain features to obtain a trained classification model, including:

[0030] Based on the first and second loss functions, the source domain features and target domain features after transfer are aligned at their edge distributions.

[0031] Based on the third loss function, the transferred source domain features and target domain features are conditionally aligned.

[0032] Based on the fourth loss function, the source domain features after aligning the target domain features with the edge distribution and the conditional distribution are classified to obtain the first classification result.

[0033] Based on the first classification result, an initial model is trained to obtain a trained classification model.

[0034] Based on the sixth possible implementation of the first aspect, in the seventh possible implementation of the first aspect, the second loss function, the third loss function, and the fourth loss function can form a collaborative optimization function.

[0035] A second aspect of this application provides an intent recognition method, including:

[0036] Acquire the electroencephalogram (EEG) signals of the target subject.

[0037] Based on the classification model, the EEG signals of the target object are classified to obtain a second classification result; the classification model is a classification model trained according to the EEG signal classification model training method provided in the first aspect above.

[0038] Based on the second classification results, the intention corresponding to the target object's EEG signals is identified.

[0039] A third aspect of this application provides a training device for an electroencephalogram (EEG) signal classification model, comprising:

[0040] The acquisition module is used to acquire the target domain features of the target domain data and the source domain features of the source domain data; the target domain data includes the EEG signal of the target object, and the source domain data includes the EEG signal of the first object, which includes objects other than the target object.

[0041] The determination module is used to determine the global temporal dependency of the target domain features based on the attention mechanism. The global temporal dependency is the correlation data between the feature data of the target domain features at various locations in the temporal domain.

[0042] The migration module is used to migrate features that are globally time-domain dependent to features in the source domain, resulting in migrated features in the source domain.

[0043] The training module is used to train the initial model based on the target domain features and the transferred source domain features to obtain the trained classification model.

[0044] A fourth aspect of this application provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the EEG signal classification model training method provided in the first aspect, or the steps of the intent recognition method provided in the second aspect.

[0045] A fifth aspect of this application provides a computer-readable storage medium storing a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the EEG signal classification model training method provided in the first aspect, or the steps of the intent recognition method provided in the second aspect.

[0046] A sixth aspect of this application provides a computer program product that, when run on a terminal, causes the terminal to execute the steps of the EEG signal classification model training method provided in the first aspect, or to execute the steps of the intent recognition method provided in the second aspect.

[0047] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0048] The beneficial effects of the embodiments in this application compared with the prior art are:

[0049] After acquiring the target domain features and source domain features, the global temporal dependency of the target domain features is transferred to the source domain features. This promotes the alignment of source and target domain features, enabling the transferred source domain features to be used to train the initial model and obtain a classification model adapted to the target object. Specifically, by determining the global temporal dependency of the target domain features based on an attention mechanism, and thereby adjusting the feature distribution of the source domain features, the effectiveness of transfer learning can be improved. The EEG signal classification model training method provided in this application reduces the requirement for target object EEG signal data, improves the training efficiency of the classification model, and reduces the preparation time before using the brain-computer interface system, because the EEG signal data of the first object and the target object can be used together to train the initial model. When the classification model is applied to a new target object, the prior training data can still be used to adjust and train the classification model to obtain a classification model adapted to the new target object. Therefore, the trained classification model has better versatility. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating a brainwave signal classification model training method provided in an embodiment of this application;

[0052] Figure 2 This is a flowchart illustrating a brainwave signal classification model training method provided in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of the structure of a parallel convolutional neural network provided in an embodiment of this application;

[0054] Figure 4 This is a flowchart illustrating a brainwave signal classification model training method provided in an embodiment of this application;

[0055] Figure 5 This is a flowchart illustrating a brainwave signal classification model training method provided in an embodiment of this application;

[0056] Figure 6 This is a schematic diagram of the structure of a global adaptor provided in an embodiment of this application;

[0057] Figure 7 This is a flowchart illustrating a brainwave signal classification model training method provided in an embodiment of this application;

[0058] Figure 8 This is a flowchart illustrating a brainwave signal classification model training method provided in an embodiment of this application;

[0059] Figure 9 This is a flowchart illustrating a brainwave signal classification model training method provided in an embodiment of this application;

[0060] Figure 10 This is a schematic diagram of the training process of the EEG signal classification model training method provided in an embodiment of this application;

[0061] Figure 11 This is a schematic flowchart of an intent recognition method provided in an embodiment of this application;

[0062] Figure 12 This is a schematic diagram of the structure of an EEG signal classification model training device provided in an embodiment of this application;

[0063] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0064] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0065] Brain-computer interface (BCI) refers to the connection created between the brain of a person or animal and an external device to enable information exchange between the brain and the external device.

[0066] Brain-computer interfaces (BCIs) can be divided into two types: non-invasive and implantable. Non-invasive BCIs are devices worn on the outside of the head, without needing to penetrate into the tissues below the skull. Implantable BCIs, on the other hand, need to penetrate into the tissues below the skull, and can even reach into the cerebral cortex.

[0067] The advantages of non-invasive brain-computer interfaces are lower cost and no trauma or surgical risks. The advantages of implantable brain-computer interfaces are higher spatiotemporal resolution and greater information content in the recorded electroencephalogram (EEG) signals. This application does not specifically limit the type of brain-computer interface; those skilled in the art can choose according to their needs.

[0068] Brain-computer interface (BCI) systems have been widely used in various fields. For example, when applied to the gaming industry, BCI systems can control virtual characters on the game screen to perform corresponding actions, such as walking, jumping, and turning, based on the user's brain signals.

[0069] Brain-computer interface (BCI) systems can help users control devices to perform tasks using their brain signals. The working principle is that after the BCI collects brain signals, it classifies these signals and identifies the corresponding control intentions based on the classification results, thereby sending corresponding control commands to the device.

[0070] Therefore, determining the category of EEG signals in order to accurately analyze the corresponding control intentions is a very important issue.

[0071] Therefore, machine learning methods can be used to train a classification model for determining the category of EEG signals.

[0072] Machine learning studies how computers can simulate human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve themselves. Therefore, in this embodiment, the classification model refers to predicting the category of new EEG signal data by learning and training existing EEG signal data.

[0073] However, because EEG signals themselves contain a significant amount of noise, a large amount of target user data is typically required to calibrate the initial model in order to obtain a classification model with good classification performance. Furthermore, individual differences due to physiological states and environments mean that EEG signal data from other users cannot be directly applied to calibrate the classification model for the target user. Direct application of such data may even reduce the effectiveness of the classification model. This results in excessively long preparation times before using the brain-computer interface system, significantly impacting its practicality.

[0074] Based on the above analysis, due to the significant differences in EEG signals among different users, a classification model trained on data from a single user is difficult to apply to classifying the EEG signals of other users. This results in poor versatility of the classification model in practical applications, making it unsuitable as a universal classification model for multiple users, and also leading to low training efficiency.

[0075] In view of this, embodiments of this application provide a method for training an EEG signal classification model. Through transfer learning, the parameters of a pre-trained model are transferred to a new model to aid in its training. Specifically, EEG signal data from other users are transferred to a new model to assist in its training, thereby obtaining a classification model adapted to the target user.

[0076] In this embodiment, the global temporal dependency of the target user's EEG signal data can be determined based on an attention mechanism neural network. Global temporal dependency refers to the correlation data between feature data at various locations within the target user's EEG data in the time domain. This global temporal dependency characteristic is then transferred to the EEG signal data of other users. This allows the transferred EEG signal data of other users to participate in training the initial model for the target user, resulting in a classification model adapted to the target user.

[0077] It should be understood that, relative to the target user, the initial model refers to a classification model that has not undergone classification training or has not undergone complete classification training.

[0078] Because the distribution differences of EEG signal characteristics at various locations in the time domain are more pronounced than in the spatial domain, the embodiments of this application transfer globally time-domain dependent characteristics. The resulting classification model achieves higher classification accuracy.

[0079] The EEG signal classification model training method provided in this application improves the training efficiency of the classification model because it can use EEG signal data from other users and the target user to jointly train the initial model. Since the initial model can be trained using EEG signal data from other users, when the classification model is applied to a new target user, the prior training data can still be used to adjust and train the classification model to obtain a classification model adapted to the new target user. Therefore, the trained classification model has better versatility.

[0080] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0081] Figure 1 A flowchart illustrating a brainwave signal classification model training method according to an embodiment of this application is shown below:

[0082] S101. Obtain the target domain features of the target domain data and the source domain features of the source domain data. The target domain data includes the EEG signals of the target object, and the source domain data includes the EEG signals of the first object.

[0083] It should be noted that since both humans and animals can emit brainwave signals, the target object and the first object here can refer to either a human or an animal; this application embodiment does not specifically limit this. For ease of explanation, in this application embodiment, both the target object and the first object refer to a human.

[0084] Because transfer learning is needed to transfer relevant features from the target domain data to the source domain data, allowing the transferred source domain data to participate in training the initial model and improve training efficiency, the first object should include objects other than the target object.

[0085] It should be understood that in the field of machine learning technology, raw data generally needs to undergo feature extraction before it can be used as training data and input into the initial model for training. Therefore, before training, it is necessary to obtain the feature data of the raw data. This feature data is then used as the training data. In this embodiment of the application, it is necessary to obtain the target domain features of the target domain data and the source domain features of the source domain data.

[0086] Furthermore, the original data may contain some interference, such as noise and artifacts. Therefore, before obtaining the feature data of the original data, it is necessary to preprocess the original data, that is, remove the interference in the target domain data and / or the source domain data.

[0087] As an example rather than a limitation, the preprocessing procedure may include three parts: truncation, bandpass filtering, and normalization.

[0088] Segmentation refers to dividing an EEG signal into multiple segments according to a time range defined by the experimental paradigm, and simultaneously converting the data channels of each segment into one dimension. Each segment corresponds to a one-dimensional EEG data point. Therefore, the data format of the truncated EEG signal includes both channel and time dimensions.

[0089] It should be noted that an experimental paradigm is a relatively fixed experimental procedure. This includes the purpose, specific process, and methods of the experiment.

[0090] Bandpass filtering refers to filtering out high-frequency and low-frequency noise irrelevant to classification training, as well as noticeable artifacts. As an example, and not a limitation, a bandpass filter can be used to achieve this purpose. For instance, a sixth-order Chebyshev bandpass filter can be used to filter data to 4-40 Hz, including EEG rhythms. β wave and μ Wave.

[0091] Standardization refers to eliminating non-stationary disturbances caused by outliers or abnormal data samples. As an example, and not a limitation, the following formula can be used to standardize data:

[0092] (1)

[0093] in, x i and x o These represent the input and output, respectively. μ and σ 2 These represent the mean and variance of the training dataset, respectively.

[0094] After the above preprocessing is performed, step S101 is performed: obtain the target domain features of the target domain data and the source domain features of the source domain data.

[0095] In some embodiments, a feature extractor can be used to extract target domain features from the target domain data and source domain features from the source domain data.

[0096] It should be understood that in this embodiment, there are no specific limitations on how to extract target domain features from target domain data or how to extract source domain features from source domain data. This is because the purpose of step S101 is to obtain target domain features and source domain features, without considering how to extract the features.

[0097] As an example, and not a limitation, based on the characteristics of EEG data, spatiotemporal depth representations of target domain and source domain data can be obtained along the time and spatial channel dimensions. The spatiotemporal depth representation of the target domain data is used as the target domain feature. The spatiotemporal depth representation of the source domain data is used as the source domain feature.

[0098] like Figure 2 As shown, in some embodiments, step S101 may include steps S201 to S203.

[0099] S201. Perform temporal convolution and spatial convolution on the first data respectively to obtain the first temporal feature and the first spatial feature. The first data is either target domain data or source domain data.

[0100] As an example rather than a limitation, the feature extractor includes one-dimensional convolutional layers along the time and spatial channel dimensions.

[0101] It should be noted that temporal convolution and spatial convolution can be connected sequentially or in parallel.

[0102] For example, in a sequential convolution, the first data is first convolved in the temporal domain, and then the first data after temporal convolution is convolved in the spatial domain to obtain the spatiotemporal depth representation of the first data. However, the first data loses a significant amount of feature information during this sequential convolution calculation.

[0103] like Figure 3 As shown, to avoid information loss caused by the serial convolution calculation process and to obtain better temporal and spatial representations, in some embodiments, the temporal and spatial convolutions of the feature extractor are connected in parallel to perform temporal and spatial convolutions on the first data respectively.

[0104] S202. Adjust the dimensions of the first time-domain feature and the first spatial-domain feature to be consistent.

[0105] After performing temporal and spatial convolutions on the first data, we obtain the first temporal feature and the first spatial feature. Since the dimensions of the first temporal feature and the first spatial feature are inconsistent, we need to adjust their dimensions to be consistent before performing operations on them to obtain the spatiotemporal depth representation of the first data, i.e., to obtain the first feature.

[0106] S203. Perform operations on the first time-domain feature and the first spatial-domain feature with the same size to obtain the first feature. The first feature is either a target domain feature or a source domain feature.

[0107] It should be noted that the specific algorithm for this operation is not specifically limited in the embodiments of this application, and those skilled in the art can choose it as needed.

[0108] As an example and not a limitation, the operation includes, but is not limited to, addition, subtraction, multiplication, and division.

[0109] In this embodiment, addition is used to calculate the sum between the first time-domain feature and the first spatial-domain feature with the same size to obtain the first feature.

[0110] As an example, and not a limitation, the kernel size of temporal convolution is... k×1×n The kernel size of spatial domain convolution is... k×m×1 The kernel size of the feature-adjusted convolution for the first temporal feature is... 1×m×1 The kernel size of the feature-adjusted convolution for the first spatial domain features is... 1×1×n .in, k, m, n All are integers. n The value can be selected according to the actual situation. m The number of channels for inputting EEG signal data is usually taken as the reference.

[0111] in, Figure 3 The batch normalization and activation functions shown can be used to improve fitting ability. Normalization is a way to simplify calculations, transforming dimensional expressions into dimensionless expressions, i.e., scalar expressions. Batch normalization refers to the meaning of batch normalization. Activation functions are functions that operate on neurons in a neural network, responsible for mapping the neuron's input to its output. Because activation functions can introduce nonlinear factors, they improve fitting ability.

[0112] Technicians can choose specific batch normalization methods and activation functions according to actual needs, and this application does not impose specific limitations on them.

[0113] In this embodiment, parallel convolution in the temporal and spatial domains is employed to obtain more accurate temporal or spatial representations. This results in a more accurate spatiotemporal depth representation of the first data, thereby improving the training quality of the classification model.

[0114] S102. Based on the attention mechanism, determine the global temporal dependency of the target domain features. The global temporal dependency is the correlation data between feature data at various locations in the time domain.

[0115] S103. Transfer the global time-domain dependent features to the source domain features to obtain the transferred source domain features.

[0116] The attention mechanism originates from research on human vision. In cognitive science, due to the bottleneck of information processing, humans selectively focus on a portion of all information while ignoring other visible information. This mechanism is commonly referred to as the attention mechanism.

[0117] Based on the above description, in this embodiment, the global temporal dependency of the target domain features is determined based on the attention mechanism, which can improve the effect of transfer learning and promote the alignment of source domain features and target domain features.

[0118] like Figure 4 As shown, in some embodiments, step S102 includes steps S401 to S404.

[0119] S401. Convert the feature channels of the target domain features into one dimension to obtain the first one-dimensional data.

[0120] Based on the characteristics of EEG data, the target domain features can contain multiple feature channels, and each feature channel corresponds to multiple feature data. That is, in each feature channel, its multiple feature data correspond to feature data at multiple locations in the time domain.

[0121] Therefore, in step S401, the feature channels of the target domain feature are converted into one dimension, which is equivalent to adjusting the number of feature channels to one, that is, arranging all feature data in one dimension.

[0122] It should be noted that the embodiments of this application do not impose specific limitations on the arrangement order of feature data in one dimension, and those skilled in the art can make adjustments as needed.

[0123] It should be noted that, in some embodiments, each element in the first one-dimensional data corresponds to the feature data of the target domain feature at each position in the time domain.

[0124] S402. Perform a linear transformation on the first one-dimensional data to obtain the first query and the first key.

[0125] As an example, and not a limitation, a first linear transformation is performed on the first one-dimensional data to obtain the first query. A second linear transformation is performed on the first one-dimensional data to obtain the first key.

[0126] It should be noted that the attention mechanism includes three parameters: query, key, and value. In this embodiment, the transferred source domain features are obtained by calculating the query, key, and source domain feature values ​​corresponding to the target domain features.

[0127] S403. Calculate the dot product between the first query and the transposed first key to obtain the global relevance matrix of the target domain features.

[0128] In this embodiment of the application, the global relevance matrix of the target domain features can be obtained by calculating the dot product between the first query and the transposed first key.

[0129] S404. Perform scaling and normalization on the global correlation matrix to obtain the correlation weight matrix of the target domain features. The correlation weight matrix is ​​the global time domain dependency.

[0130] Before transferring the global correlation matrix, it needs to be scaled and normalized to unify the dimensions and simplify calculations. After scaling and normalization, the correlation weight matrix of the target domain features is obtained. The correlation weight matrix represents the global time-domain dependency.

[0131] like Figure 5 As shown, in some embodiments, step S103 includes steps S501 to S503.

[0132] S501. Convert the feature channels of the source domain features into one dimension to obtain the second one-dimensional data.

[0133] In order to transfer the global time-domain dependent features to the source domain features, the feature channels of the source domain features also need to be converted into one dimension to obtain the second one-dimensional data.

[0134] It should be noted that, in some embodiments, each element in the second one-dimensional data corresponds to the feature data of the source domain features at various positions in the time domain.

[0135] S502. Perform a linear transformation on the second one-dimensional data to obtain the first value.

[0136] S503. Calculate the dot product of the first value and the correlation weight matrix to obtain the source domain features after transfer.

[0137] Based on the above description, and using the attention mechanism, the dot product of the first value and the relevance weight matrix can be calculated to obtain the transferred source domain features.

[0138] As an example rather than a limitation, such as Figure 6 The diagram illustrates the process of transferring the global temporal dependency of features in the target domain to features in the target domain. As an example, and not a limitation, this process can be expressed by the following formula:

[0139] (2)

[0140] in, Q t This represents the first query corresponding to the target domain features.K t This represents the first key corresponding to the target domain feature. V s This represents the first value corresponding to the source domain feature. K t T express K t The transpose of . Softmax This represents the exponential normalization function. d express Q t and K t The length.

[0141] As mentioned above, in Figure 4 and Figure 5 In the illustrated embodiment, each element in the first one-dimensional data corresponds to the feature data of the target domain feature at each position in the time domain. Each element in the second one-dimensional data corresponds to the feature data of the source domain feature at each position in the time domain. To improve the accuracy of solving the global time-domain dependency and reduce the complexity of the solution process, the first and second one-dimensional data can be divided into multiple one-dimensional data.

[0142] For example, in some embodiments, such as Figure 7 As shown, step S401 includes the following steps:

[0143] S701. Divide the target domain features into multiple first slice features along the time dimension.

[0144] S702. Convert the feature channels of each first slice feature into one dimension to obtain multiple third one-dimensional data corresponding to each first slice feature. The first one-dimensional data includes these multiple third one-dimensional data.

[0145] Therefore, substituting steps S701 and S702 into... Figure 4 The illustrated embodiment. Steps S402 to S404 include the following steps:

[0146] Perform linear transformations on multiple third-dimensional data to obtain the second query and second key corresponding to each third-dimensional data.

[0147] Calculate the dot product between the second query and the transposed second key corresponding to each third one-dimensional data to obtain the first correlation matrix corresponding to each first slice feature.

[0148] Each first correlation matrix is ​​scaled and normalized to obtain the first weight matrix corresponding to each first slice feature. These first weight matrices then form the correlation weight matrix.

[0149] For example, in some embodiments, such as Figure 8 As shown, step S501 includes the following steps:

[0150] S801. Divide the source domain features into multiple second slice features along the time dimension.

[0151] S802. Convert the feature channels of each second slice feature into one dimension to obtain multiple fourth one-dimensional data corresponding to each second slice feature. The second one-dimensional data includes these multiple fourth one-dimensional data.

[0152] Therefore, substituting steps S801 and S802 into... Figure 5 The illustrated embodiment. Steps S502 to S503 include the following steps:

[0153] By performing linear transformations on multiple fourth-dimensional data, the second value corresponding to each fourth-dimensional data is obtained.

[0154] The dot product between the second value corresponding to each fourth one-dimensional data point and each first weight matrix is ​​calculated to obtain multiple transferred second slice features. These multiple transferred second slice features constitute the transferred source domain features.

[0155] In this embodiment, the target domain features are divided into multiple first slice features along the time dimension, and then the first weight matrix corresponding to each first slice feature is calculated. These first weight matrices together form the correlation weight matrix. This process improves the accuracy of the calculated correlation weight matrix while reducing computational complexity.

[0156] In some embodiments, in step S102, the global temporal dependency of the target domain features can be determined based on a multi-head attention mechanism to improve the accuracy of global temporal dependency calculation.

[0157] Multi-head attention mechanism: This refers to dividing the input of the attention mechanism into multiple equal parts, called heads; implementing the attention mechanism on each head; and concatenating the results of the attention mechanisms of each head to obtain the final output.

[0158] This process can be represented by the following formula:

[0159] (3)

[0160] (4)

[0161] in, h Indicates the number of multiple heads in the partition. , , These represent the linear transformation relationships used to obtain the query, key, and value, respectively. l Indicates the index of the header.x t For target domain data, x s This is source domain data.

[0162] S104. Based on the target domain features and the transferred source domain features, the initial model is trained to obtain the trained classification model.

[0163] In this embodiment, the initial model can be trained using both the EEG signal data of the first object and the target object. Therefore, a smaller amount of EEG signal data from the target object, combined with a larger amount of EEG signal data from the first object, can be used to train the initial model, thereby improving the training efficiency of the classification model. When the classification model is applied to a new target object, the prior training data can still be used to adjust and train the classification model to obtain a classification model adapted to the new target object. Therefore, the trained classification model has better versatility.

[0164] like Figure 9 As shown, in some embodiments, step S104 includes the following steps S901 to S904.

[0165] S901. Based on the first loss function and the second loss function, the source domain features and target domain features after transfer are aligned at their edge distributions.

[0166] As an example, and not a limitation, to ensure that the transferred source domain features are marginally aligned with the target domain features, the first loss function can be set in the manner of WGAN (Wasserstein Generative Adversarial Networks). The first loss function is the objective function of the discriminator.

[0167] The first loss function is:

[0168] (5)

[0169] (6)

[0170] (7)

[0171] in, E Indicates the expected value; x s For source domain data, H s The range of the source domain data; x t For target domain data, H t The range of data in the target domain; GP This represents a gradient penalty, used to improve convergence speed; This is a hyperparameter; α is a random value between 0 and 1. F, A, D These represent the processes of feature extraction, global temporal dependency feature transfer, and training the initial model, respectively.

[0172] To establish an adversarial learning relationship, the constraint on the feature extractor and the global adaptor corresponding to the objective function of the discriminator is the second loss function.

[0173] The second loss function is:

[0174] (8)

[0175] After the feature alignment process described above, both the target domain features and the aligned source domain features can be used to train a classifier for the target object.

[0176] Under alternating training, the feature extractor and global adaptor tend to find features that are distributed in the source and target domains, that is, gradually align the distribution pairs of the source and target domains.

[0177] S902. Based on the third loss function, the transferred source domain features and target domain features are conditionally aligned.

[0178] As an example rather than a limitation, the third loss function is:

[0179] (9)

[0180] Here, ct represents the feature centers of different categories of EEG signals in the target domain data.

[0181] Therefore, under the constraint of the third loss function, the source domain feature distributions of different categories become closer to the target domain features of the corresponding categories. At the same time, intra-class differences decrease, while inter-class differences increase.

[0182] S903. Based on the fourth loss function, the source domain features after aligning the target domain features with the edge distribution and the conditional distribution are classified to obtain the first classification result.

[0183] As an example rather than a limitation, the fourth loss function is:

[0184] (10)

[0185] in, N s The amount of source domain data, N t The number of data in the target domain. M The number of categories of EEG signals. y and These represent the true label and the predicted label for the classification, respectively. i, c Integers respectively.

[0186] As an example and not a limitation, the second, third, and fourth loss functions mentioned above can be combined to form the following collaborative optimization loss function.

[0187] (11)

[0188] in, , represents the weights of the second and third loss functions, respectively, and is a hyperparameter.

[0189] In this embodiment, based on the first loss function and the second loss function, under the constraint of adversarial learning, the source domain features and target domain features after transfer are aligned at their edge distributions.

[0190] In this embodiment, the third loss function is used to align the conditional distribution of the source domain features with that of the target domain features after migration, which can further eliminate the conditional distribution differences between the source domain data and the target domain data.

[0191] S904. Based on the first classification result, train the initial model to obtain the trained classification model.

[0192] Based on the above description, the initial model can be trained under alternating adversarial learning to obtain the trained classification model.

[0193] In summary, Figure 10 The diagram illustrates the training process of an EEG signal classification model training method provided in an embodiment of this application.

[0194] In this embodiment, after the target domain data is input into the feature extractor, the target domain features are output. After the source domain data is input into the feature extractor, the source domain features are output. When the target domain features are input into the global adaptor, the global temporal dependency of the target domain features is obtained based on the attention mechanism, and this global temporal dependency is transferred to the source domain features.

[0195] Therefore, the global adaptor outputs target domain features and transferred source domain features. These target domain features and transferred source domain features can be simultaneously input into both the classifier and discriminator. After adversarial learning, the trained classification model is finally obtained. Here, the adaptive center loss represents the third loss function.

[0196] Figure 11 A flowchart illustrating an embodiment of the intent recognition method provided in this application is shown. The intent recognition method includes:

[0197] S1101. Obtain the EEG signal of the target object.

[0198] S1102. Based on the classification model, classify the EEG signals of the target object to obtain a second classification result. The classification model is as follows: Figures 1 to 10 The training method for the EEG signal classification model provided in any of the embodiments shown, and the classification model obtained by training.

[0199] S1103. Based on the second classification result, identify the intention corresponding to the EEG signal of the target object.

[0200] In this embodiment, the following is applied: Figures 1 to 10 The classification model trained in any of the illustrated embodiments classifies the EEG signals of the target object. Then, based on the second classification result, the intention corresponding to the target object's EEG signals can be identified.

[0201] The intent recognition method provided in this embodiment uses [a specific method / technology]. Figures 1 to 10 The classification model trained in any of the illustrated embodiments is therefore capable of accurately classifying the EEG signals of the target object. Furthermore, the short training time before use improves the practicality of this classification model.

[0202] It should be understood that, provided there are no logical conflicts, the above-described embodiments can be combined and implemented to adapt to actual application needs. These combined embodiments or implementation schemes are still within the scope of protection of this application.

[0203] Corresponding to the EEG signal classification model training method described in the above embodiments, Figure 12 A schematic diagram of the structure of an EEG signal classification model training device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0204] Please see Figure 12 As shown, the EEG signal classification model training device includes:

[0205] The acquisition module 121 is used to acquire target domain features of the target domain data and source domain features of the source domain data. The target domain data includes the EEG signals of the target object, and the source domain data includes the EEG signals of a first object, which includes objects other than the target object.

[0206] Module 122 is used to determine the global temporal dependency of the target domain features based on an attention mechanism. The global temporal dependency is the correlation data between feature data at various locations in the time domain.

[0207] The migration module 123 is used to migrate the features that are globally time-domain dependent to the features of the source domain, so as to obtain the migrated features of the source domain.

[0208] Training module 124 is used to train the initial model based on the target domain features and the transferred source domain features to obtain the trained classification model.

[0209] In some embodiments, the determining module 122 includes:

[0210] The first transformation unit is used to convert the feature channels of the target domain features into one dimension to obtain the first one-dimensional data.

[0211] The first transformation unit is used to perform a linear transformation on the first one-dimensional data to obtain the first query and the first key.

[0212] The first computational unit is used to calculate the dot product between the first query and the transposed first key to obtain the global relevance matrix of the target domain features.

[0213] The second computational unit is used to scale and normalize the global correlation matrix to obtain the correlation weight matrix of the target domain features. The correlation weight matrix is ​​a global time-domain dependency.

[0214] In some embodiments, the migration module 123 includes:

[0215] The second conversion unit is used to convert the feature channels of the source domain features into one dimension to obtain the second one-dimensional data.

[0216] The second transformation unit is used to perform a linear transformation on the second one-dimensional data to obtain the first value.

[0217] The third computational unit is used to calculate the dot product of the first value and the correlation weight matrix to obtain the transferred source domain features.

[0218] In some embodiments, the first conversion unit includes:

[0219] The first partitioning unit is used to divide the target domain features into multiple first slice features along the time dimension.

[0220] The third conversion unit is used to convert the feature channels of each first slice feature into one dimension to obtain multiple third one-dimensional data, and the first one-dimensional data contains the multiple third one-dimensional data.

[0221] In some embodiments, the second conversion unit includes:

[0222] The second partitioning unit is used to divide the source domain features into multiple second slice features along the time dimension.

[0223] The fourth conversion unit is used to convert the feature channels of each second slice feature into one dimension to obtain multiple fourth one-dimensional data, and the second one-dimensional data contains the multiple fourth one-dimensional data.

[0224] In some embodiments, the acquisition module 121 includes:

[0225] The first convolutional unit is used to perform temporal convolution and spatial convolution on the first data to obtain the first temporal feature and the first spatial feature. The first data is either target domain data or source domain data.

[0226] The second convolutional unit is used to adjust the size of the first temporal feature and the first spatial feature to be consistent.

[0227] The fourth computational unit is used to perform calculations based on the first time-domain feature and the first spatial-domain feature with consistent dimensions to obtain the first feature. The first feature is either a target domain feature or a source domain feature.

[0228] The process by which each module in the EEG signal classification model training device provided in this application implements its respective function can be specifically referred to the foregoing. Figure 1 The descriptions of the embodiments shown and other related method embodiments are not repeated here.

[0229] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0230] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0231] It should be understood that, when used in this application specification and the appended claims, the terms "comprising," "including," etc., indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0232] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. It should also be understood that although the terms "first," "second," etc., are used to describe various elements in some embodiments of this application, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first computing unit may be named a second computing unit, and similarly, a second computing unit may be named a first computing unit, without departing from the scope of the various described embodiments. Both the first computing unit and the second computing unit are computing units, but they are not the same computing unit.

[0233] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0234] The EEG signal classification model training method provided in this application embodiment can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application embodiment does not impose any restrictions on the specific type of electronic device.

[0235] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 13 As shown, the electronic device 13 of this embodiment includes: at least one processor 130 ( Figure 13 (Only one is shown in the image), memory 131, which stores a computer program 132 that can run on processor 130. When processor 130 executes computer program 132, it implements the steps in the above-described embodiments of the various EEG signal classification model training methods, for example... Figure 1 Steps S101 to S104 as shown; or Figure 11 The steps S1101 to S1103 are shown. Alternatively, when the processor 130 executes the computer program 132, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 12 The functions of modules 121 to 124 are shown.

[0236] Electronic device 13 may be a desktop computer, laptop, handheld computer, or cloud server, etc. Electronic device 13 may include, but is not limited to, a processor 130 and a memory 131. Those skilled in the art will understand that... Figure 13This is merely an example of electronic device 13 and does not constitute a limitation on electronic device 13. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 13 may also include input transmitting devices, network access devices, buses, etc.

[0237] The processor 130 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0238] In some embodiments, memory 131 may be an internal storage unit of electronic device 13, such as a hard disk or memory of electronic device 13. Memory 131 may also be an external storage device of electronic device 13, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on electronic device 13. Memory 131 may also include both internal and external storage units of electronic device 13. Memory 131 is used to store operating system, applications, bootloader, data, and other programs, such as the program code of computer program 132. Memory 131 may also be used to temporarily store data that has been sent or will be sent.

[0239] Furthermore, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. In the various embodiments of this application, each functional unit can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0240] This application also provides an electronic device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, it causes the electronic device to perform the steps in the above-described method embodiments.

[0241] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0242] This application provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0243] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0244] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0245] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0246] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0247] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for training a brainwave signal classification model, characterized in that, include: The target domain features of the target domain data and the source domain features of the source domain data are obtained; the target domain data includes the EEG signal of the target object, and the source domain data includes the EEG signal of a first object, wherein the first object includes objects other than the target object. The feature channels of the target domain feature are converted into one dimension to obtain the first one-dimensional data. Perform a linear transformation on the first one-dimensional data to obtain the first query and the first key; Calculate the dot product between the first query and the transposed first key to obtain the global relevance matrix of the target domain features; The global correlation matrix is ​​scaled and normalized to obtain the correlation weight matrix of the target domain features, and the correlation weight matrix is ​​a global time domain dependency. The global time-domain dependent features are transferred to the source domain features to obtain the transferred source domain features; Based on the target domain features and the transferred source domain features, the initial model is trained to obtain the trained classification model.

2. The training method according to claim 1, characterized in that, The step of transferring the globally time-domain dependent features to the source domain features to obtain the transferred source domain features includes: The feature channels of the source domain features are converted into one dimension to obtain the second one-dimensional data. Perform a linear transformation on the second one-dimensional data to obtain the first value; The dot product of the first value and the correlation weight matrix is ​​calculated to obtain the transferred source domain features.

3. The training method according to claim 1, characterized in that, The step of converting the feature channels of the target domain features into one dimension to obtain the first one-dimensional data includes: The target domain features are divided into multiple first slice features along the time dimension; Each feature channel of the first slice is converted into one dimension to obtain multiple third one-dimensional data, and the first one-dimensional data contains multiple third one-dimensional data.

4. The training method according to claim 2, characterized in that, The step of converting the feature channels of the source domain features into one dimension to obtain the second one-dimensional data includes: The source domain features are divided into multiple second slice features along the time dimension; Each feature channel of the second slice feature is converted into one dimension to obtain multiple fourth one-dimensional data, and the second one-dimensional data contains multiple fourth one-dimensional data.

5. The training method according to claim 1, characterized in that, The acquisition of target domain features of target domain data and source domain features of source domain data includes: The first data is subjected to temporal convolution and spatial convolution respectively to obtain first temporal features and first spatial features; the first data is the target domain data or the source domain data. Adjust the dimensions of the first temporal feature and the first spatial feature to be consistent; The first feature is obtained by performing calculations based on the first temporal feature and the first spatial feature with consistent size; the first feature is either the target domain feature or the source domain feature.

6. The training method according to claim 1, characterized in that, The step of training the initial model based on the target domain features and the transferred source domain features to obtain the trained classification model includes: Based on the first loss function and the second loss function, the source domain features after transfer are aligned with the target domain features in terms of edge distribution; Based on the third loss function, the transferred source domain features and target domain features are conditionally aligned. Based on the fourth loss function, the source domain features after alignment with the target domain features and the edge distribution and the conditional distribution are classified to obtain the first classification result; Based on the first classification result, the initial model is trained to obtain the trained classification model; The third loss function is: ， E Indicates the expected value. x s For the source domain data, H s The range of the source domain data. x t For the target domain data, H t The range of the target domain data. F, A These respectively represent the feature extraction process and the transfer process of the globally temporally dependent characteristics. c t These represent the feature centers of different categories of EEG signals in the target domain data.

7. An intent recognition method, characterized in that, include: Acquire the electroencephalogram (EEG) signals of the target subject; Based on the classification model, the EEG signals of the target object are classified to obtain a second classification result; The classification model is a classification model trained by the EEG signal classification model training method according to any one of claims 1 to 6; Based on the second classification result, the intention corresponding to the EEG signal of the target object is identified.

8. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the steps of the EEG signal classification model training method according to any one of claims 1 to 6, or to implement the steps of the intention recognition method according to claim 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the EEG signal classification model training method according to any one of claims 1 to 6, or the steps of the intent recognition method according to claim 7.

Images