Electroencephalogram signal recognition method of domain decoupling electroencephalogram representation learning model based on hybrid experts

Abstract

The invention provides an electroencephalogram signal recognition method of a domain decoupling electroencephalogram representation learning model based on hybrid experts, and the method comprises the steps: carrying out the spatial domain coding of sample electroencephalogram data, and obtaining spatial domain feature electroencephalogram data; performing frequency domain coding on the spatial domain feature electroencephalogram data to obtain sample electroencephalogram feature data; performing multi-head self-attention calculation on the sample electroencephalogram feature data to obtain sample electroencephalogram attention feature data; performing hybrid expert domain decoupling processing on the sample electroencephalogram attention feature data to obtain sample electroencephalogram reconstruction data; and according to the sample electroencephalogram reconstruction data and the sample electroencephalogram feature data, performing parameter adjustment on the electroencephalogram representation learning model to obtain a trained electroencephalogram representation learning model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically to an electroencephalogram (EEG) signal recognition method based on a domain-decoupled EEG representation learning model of mixed experts. Background Art

[0002] EEG signals are physiological signals that can reflect brain activity and play an important role in many neuroscience applications, such as emotion recognition, motor imagery, mental workload detection, sleep stage detection, epilepsy analysis, decision prediction, etc.

[0003] Currently, many deep learning models and feature extraction methods are applied in the above-mentioned fields. However, in related technologies, existing EEG models use parameters to model heterogeneous EEG data, ignoring the potential differences and conflicts between EEG signals between different tasks. EEG signals often have significant differences in different experimental subjects, data sets and tasks. Current EEG models fail to fully unleash the potential of EEG representation, resulting in a large difference between the predicted EEG data and the actual EEG data. Summary of the invention

[0004] In view of the above problems, the present disclosure provides an EEG signal recognition method based on a domain decoupled EEG representation learning model of mixed experts.

[0005] According to the first aspect of the present disclosure, a method for EEG signal recognition based on a hybrid expert domain decoupling EEG representation learning model is provided, including: performing spatial domain encoding on the above-mentioned sample EEG data to obtain spatial domain feature EEG data; performing frequency domain encoding on the above-mentioned spatial domain feature EEG data to obtain sample EEG feature data; performing multi-head self-attention calculation on the above-mentioned sample EEG feature data to obtain sample EEG attention feature data; performing hybrid expert domain decoupling processing on the above-mentioned sample EEG attention feature data to obtain sample EEG reconstruction data; and adjusting parameters of the EEG signal representation learning model according to the above-mentioned sample EEG reconstruction data and the above-mentioned sample EEG feature data to obtain a trained EEG signal representation learning model.

[0006] According to an embodiment of the present disclosure, the above-mentioned mixed expert domain decoupling processing is performed on the above-mentioned sample EEG attention feature data to obtain sample EEG reconstruction data, including: determining at least one target-specific expert weight based on the above-mentioned sample EEG attention feature data and the first routing weight, wherein each of the above-mentioned target-specific expert weights represents the activation probability of the corresponding target-specific expert module, each of the above-mentioned target-specific expert modules is determined from a specific expert module group based on at least one of the above-mentioned target-specific expert weights, and each of the above-mentioned target-specific expert modules is used to extract specific features of the above-mentioned sample EEG attention feature data; determining at least one shared expert weight based on the above-mentioned sample EEG attention feature data and the second routing weight, wherein each of the above-mentioned shared expert modules is used to extract common features of the above-mentioned sample attention feature data; obtaining specific expert prediction data of the sample EEG signal based on at least one of the above-mentioned target-specific expert weights and the above-mentioned sample EEG attention feature data; obtaining shared expert prediction data of the sample EEG signal based on at least one of the above-mentioned shared expert weights and the above-mentioned sample attention feature data; obtaining the above-mentioned sample EEG reconstruction data based on the above-mentioned specific expert prediction data and the above-mentioned shared expert prediction data.

[0007] According to an embodiment of the present disclosure, the above-mentioned method of determining at least one target specific expert weight based on the above-mentioned sample EEG attention feature data and the first routing weight includes: obtaining the specific expert weight corresponding to each specific expert module in the above-mentioned specific expert module group based on the above-mentioned sample EEG attention feature data and the above-mentioned first routing weight; sorting the above-mentioned specific expert weights based on preset sorting rules to obtain sorted specific expert weights; and selecting a preset number of specific expert weights that meet the preset selection rules from the above-mentioned sorted specific expert weights as the target specific expert weights.

[0008] According to an embodiment of the present disclosure, the specific expert weight, the reconstruction data and the sample EEG attention feature data are determined;

[0009] According to the above-mentioned specific expert weights, determine the first allocation ratio and the second allocation ratio of allocating the above-mentioned sample EEG attention feature data to each of the above-mentioned specific expert modules; utilize the first loss function to obtain the first loss value according to the above-mentioned reconstructed data and the above-mentioned sample EEG attention feature data; utilize the second loss function to obtain the second loss value according to the above-mentioned first allocation ratio and the above-mentioned second allocation ratio; based on the above-mentioned first loss value and the above-mentioned second loss value, adjust the parameters of the above-mentioned trained EEG signal representation learning model to obtain the trained EEG signal representation learning model.

[0010] According to an embodiment of the present disclosure, before performing spatial domain encoding on the above-mentioned sample EEG data, the above-mentioned method also includes: acquiring original EEG signal sample data, the above-mentioned original EEG sample data including position information of the signal acquisition device on the target sample object; dividing the above-mentioned original EEG signal sample data into time periods based on a preset division time period to obtain EEG time period data; dividing the above-mentioned EEG time period data into bands based on a preset filtering algorithm to obtain EEG time-frequency data; obtaining EEG time-frequency-space data based on the above-mentioned EEG time-frequency data and the above-mentioned position information; and formatting the above-mentioned EEG time-frequency-space data to obtain the above-mentioned sample EEG data.

[0011] According to an embodiment of the present disclosure, the above method also includes: inputting the second sample EEG data into an EEG signal characterization learning module in an EEG signal classification model to obtain sample EEG signal feature characterization data, wherein the above EEG signal characterization learning module is pre-loaded with an EEG signal characterization learning model, and the above EEG signal characterization learning model is obtained by the above method; inputting the above sample EEG signal feature characterization data into a classification module in the above EEG signal classification model to obtain a predicted classification result; obtaining a classification loss value based on the above predicted classification result and the actual classification result; and adjusting the parameters of the above EEG signal classification model based on the above classification loss value to obtain a trained EEG signal classification model.

[0012] According to an embodiment of the present disclosure, the above method also includes: acquiring EEG data of the target object, wherein the above EEG data includes position information of the signal acquisition device on the target object; inputting the above EEG data into an EEG signal classification model to obtain a classification result, wherein the above classification result characterizes the intended behavior or emotional state corresponding to the EEG data of the above target object, and the above EEG signal classification model is obtained by the above method.

[0013] The second aspect of the present disclosure provides an EEG signal recognition device based on a domain decoupled EEG representation learning model of a hybrid expert, including: a spatial domain encoding module, used to perform spatial domain encoding on the above-mentioned sample EEG data to obtain spatial domain feature EEG data; a frequency domain encoding module, used to perform frequency domain encoding on the above-mentioned spatial domain feature EEG data to obtain sample EEG feature data; a self-attention calculation module, used to perform multi-head self-attention calculation on the above-mentioned sample EEG feature data to obtain sample EEG attention feature data; a hybrid expert processing module, used to perform hybrid expert domain decoupling processing on the above-mentioned sample EEG attention feature data to obtain sample EEG reconstruction data.

[0014] According to an embodiment of the present disclosure, the hybrid expert processing module includes: a target-specific expert weight determination submodule, which is used to determine at least one target-specific expert weight based on the sample EEG attention feature data and the first routing weight, wherein each of the target-specific expert weights represents the activation probability of the corresponding target-specific expert module, and each of the target-specific expert modules is determined from a specific expert module group based on at least one of the target-specific expert weights, and each of the target-specific expert modules is used to extract specific features of the sample EEG attention feature data; a shared expert weight determination submodule, which is used to determine at least one shared expert weight based on the sample EEG attention feature data and the second routing weight, wherein each of the shared expert modules is used to extract common features of the sample attention feature data; a specific expert prediction submodule, which is used to obtain specific expert prediction data of a sample EEG signal based on at least one of the target-specific expert weights and the sample EEG attention feature data; a shared expert prediction submodule, which is used to obtain shared expert prediction data of a sample EEG signal based on at least one of the shared expert weights and the sample attention feature data; and a reconstruction submodule, which is used to obtain the sample EEG reconstruction data based on the specific expert prediction data and the shared expert prediction data.

[0015] According to an embodiment of the present disclosure, the above-mentioned target specific expert weight determination submodule includes: a specific expert weight determination unit, which is used to obtain the specific expert weight corresponding to each specific expert module in the above-mentioned specific expert module group based on the above-mentioned sample EEG attention feature data and the above-mentioned first routing weight; a specific expert weight sorting unit, which is used to sort the above-mentioned specific expert weights based on preset sorting rules to obtain sorted specific expert weights; and a selection unit, which is used to select a preset number of specific expert weights that meet the preset selection rules from the above-mentioned sorted specific expert weights as the target specific expert weights.

[0016] According to an embodiment of the present disclosure, the above-mentioned device also includes: an original EEG signal sample data acquisition submodule, which is used to acquire original EEG signal sample data, and the above-mentioned original EEG sample data includes the position information of the signal acquisition device on the target sample object; a time length division submodule, which is used to divide the above-mentioned original EEG signal sample data into time lengths based on a preset division time length to obtain EEG time period data; a band division submodule, which is used to divide the above-mentioned EEG time period data into bands based on a preset filtering algorithm to obtain EEG time-frequency data; a time-frequency-space data submodule, which is used to obtain EEG time-frequency-space data based on the above-mentioned EEG time-frequency data and the above-mentioned position information; a format conversion submodule, which is used to convert the format of the above-mentioned EEG time-frequency-space data to obtain the above-mentioned sample EEG data.

[0017] According to an embodiment of the present disclosure, the above-mentioned device also includes: an EEG signal learning module, which is used to input the second sample EEG data into the EEG signal characterization learning module in the EEG signal classification model to obtain sample EEG signal feature characterization data, wherein the above-mentioned EEG signal characterization learning module is pre-loaded with the EEG signal characterization learning model, and the above-mentioned EEG signal characterization learning model is obtained by the above-mentioned first aspect method; a classification input module, which is used to input the above-mentioned sample EEG signal feature characterization data into the classification module in the above-mentioned EEG signal classification model to obtain a predicted classification result; a classification loss value module, which is used to obtain a classification loss value based on the above-mentioned predicted classification result and the actual classification result.

[0018] According to an embodiment of the present disclosure, the above-mentioned device also includes: an EEG data acquisition module, used to acquire the EEG data of the target object, wherein the above-mentioned EEG data includes the position information of the signal acquisition device on the target object; an EEG signal classification model input module, used to input the above-mentioned EEG data into the EEG signal classification model to obtain a classification result, wherein the above-mentioned classification result characterizes the intended behavior or emotional state corresponding to the EEG data of the above-mentioned target object, and the above-mentioned EEG signal classification model is obtained according to the method of the above-mentioned first aspect.

[0019] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0020] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0021] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0022] According to the embodiments of the present disclosure, the sample EEG data is encoded in the spatial domain and frequency domain so as to be integrated according to the band and channel dimensions to extract effective information of different bands and channels in the sample EEG data. The mixed expert domain decoupling processing is performed through the data features in the sample EEG attention feature data. The data features in the EEG signals can be learned in a targeted manner, and the generalization ability of the model can be enhanced, so that the model can adapt to different task requirements. Further, the learning efficiency of the model is improved, and the prediction error of the EEG model is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0024] Figure 1 A diagram schematically shows an application scenario of an EEG signal recognition method and device based on a domain decoupled EEG representation learning model based on a hybrid expert according to an embodiment of the present disclosure;

[0025] Figure 2 A flowchart of an EEG signal recognition method based on a domain decoupled EEG representation learning model of mixed experts according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 3 Schematically shows a top view of a collection device and a head of a collection object according to an embodiment of the present disclosure;

[0027] Figure 4 Schematically shows a structural diagram of a hybrid expert module for performing hybrid expert domain decoupling processing according to an embodiment of the present disclosure;

[0028] Figure 5 A flowchart of constructing an EEG signal classification model according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 6 A flowchart of a method for classifying an electroencephalogram signal according to an embodiment of the present disclosure is schematically shown;

[0030] Figure 7 The structure block diagram of the EEG signal recognition device based on the domain decoupled EEG representation learning model of hybrid experts according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 8 The structure block diagram of the device for forming the EEG signal classification model according to the embodiment of the present disclosure is schematically shown;

[0032] Fig. 9 The structure block diagram of the EEG signal classification device according to the embodiment of the present disclosure is schematically shown;

[0033] Fig.10 A block diagram of an electronic device suitable for implementing an EEG signal recognition method based on a domain decoupled EEG representation learning model of a hybrid expert model according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0034] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0037] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] EEG signals are physiological signals that can reflect brain activity and play an important role in many neuroscience applications, such as emotion recognition, motor imagery, mental workload detection, sleep stage detection, epilepsy analysis, decision prediction, etc.

[0039] At present, many deep learning models and manual feature extraction methods are used in the above fields. For example, some works use differential entropy or power spectral density features and combine convolutional neural networks or long short-term memory networks to learn the subject-dependent spatiotemporal frequency information related to emotions. Some works extract features by analyzing brain connectivity features to achieve the assessment of the mental workload of specific subjects. Some works use CNN, GNN or Transformer to directly model the original EEG signals to achieve the recognition of a single task, a single data set, or even a single subject. There is also a part of work that learns potential unified representations from a large number of EEG signals through self-supervised pre-training. However, these works mainly focus on the importance of unified EEG signal input format, and use a single model to model EEG signals of all domains without distinction. Using homogeneous parameters to model heterogeneous data inevitably ignores the potential differences and conflicts between EEG signals in different tasks. In fact, EEG signals often have significant differences in different subjects, data sets and tasks, so that EEG signals with different distributions form different domains. The multi-domain coupled pre-training paradigm of current work fails to fully unleash the potential of the model to learn EEG representations, and thus their performance does not show significant advantages over domain-specific task models.

[0040] In view of this, an embodiment of the present disclosure provides an EEG signal recognition method based on a hybrid expert domain decoupling EEG representation learning model, including: performing spatial domain encoding on sample EEG data to obtain spatial domain feature EEG data; performing frequency domain encoding on spatial domain feature EEG data to obtain sample EEG feature data; performing multi-head self-attention calculation on the sample EEG feature data to obtain sample EEG attention feature data; performing hybrid expert domain decoupling processing on the sample EEG attention feature data to obtain sample EEG reconstruction data; and adjusting parameters of the EEG signal representation learning model according to the sample EEG reconstruction data and the sample EEG feature data to obtain a trained EEG signal representation learning model.

[0041] Figure 1 The application scenario diagram of the EEG signal recognition method and device based on the domain decoupled EEG representation learning model of hybrid experts according to an embodiment of the present disclosure is schematically shown.

[0042] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0043] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0044] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0045] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0046] It should be noted that the EEG signal recognition method based on the domain-decoupled EEG representation learning model of hybrid experts provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the EEG signal recognition device based on the domain-decoupled EEG representation learning model of hybrid experts provided in the embodiments of the present disclosure can generally be set in the server 105. The EEG signal recognition method based on the domain-decoupled EEG representation learning model of hybrid experts provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the EEG signal recognition device based on the domain-decoupled EEG representation learning model of hybrid experts provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0047] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0048] The following will be based on Figure 1 The scene described by Figure 2~Figure 6 The disclosed embodiment describes in detail the EEG signal recognition method based on the domain decoupled EEG representation learning model of hybrid experts.

[0049] Figure 2The flowchart of the method for EEG signal recognition based on the domain decoupled EEG representation learning model of mixed experts according to an embodiment of the present disclosure is schematically shown.

[0050] like Figure 2 As shown, the EEG signal recognition method based on the domain decoupled EEG representation learning model of mixed experts in this embodiment includes operations S210 to S250.

[0051] In operation S210, spatial domain encoding is performed on the sample EEG data to obtain spatial domain feature EEG data.

[0052] According to the embodiment of the present disclosure, the format of the sample EEG data can be, for example, , where X1 to X n It is a 3D array, which contains the time domain information, spatial domain information and frequency domain information of the EEG signal, where B represents the number of EEG (Electroencephalogram) signal bands, H represents the height of the 2D space where the EEG electrodes are arranged (the number of electrodes in the height direction of the two-dimensional grid formed by the electrodes arranged on the scalp), and W represents the width of the 2D space where the EEG electrodes are arranged (the number of electrodes in the width direction of the two-dimensional grid formed by the electrodes arranged on the scalp).

[0053] Figure 3 A top view of the head of a collection device and a collection object according to an embodiment of the present disclosure is schematically shown.

[0054] like Figure 3 As shown, the acquisition device and the overhead view of the head of the acquisition object include the head 300 of the acquisition object and the acquisition device 310, and H is 2 and W is 3 in the corresponding 3D data.

[0055] In operation S220, frequency domain encoding is performed on the spatial domain feature EEG data to obtain sample EEG feature data.

[0056] According to the embodiment of the present disclosure, a multi-layer perceptron can be used to perform spatial and frequency domain encoding on sample EEG data, and the encoded EEG data can be calculated using the following formula (1).

[0057]

[0058] in, represents the RELU activation function, represents a dimension reshaping function, which transforms the input features to reshape the multi-channel EEG signal into a two-dimensional array suitable for input into the neural network. and are two linear layers, mapping the data to and Dimension.

[0059] In operation S230, multi-head self-attention calculation is performed on the sample EEG feature data to obtain sample EEG attention feature data.

[0060] According to an embodiment of the present disclosure, before performing multi-head self-attention calculation on the sample EEG feature data, part of the data in the sample EEG feature data can be masked first. Masking is a commonly used data preprocessing technology used to hide or ignore certain data points. In the present disclosure, the sample EEG feature data can be selected for random masking to detect the learning results of the domain-decoupled EEG representation learning model based on hybrid experts.

[0061] According to the embodiment of the present disclosure, multi-head attention calculation requires dividing the input embedding (i.e., input data) into h embeddings of equal length, where h represents the number of attention heads. Each embedding is sent to a separate self-attention layer, and the output of each self-attention layer is concatenated to obtain self-attention EEG data. The self-attention EEG data can be calculated by the following formula (2).

[0062]

[0063] in, is the output data of multi-head self-attention (i.e. self-attention EEG data), represents the output of the first self-attention layer, represents the output of the hth self-attention layer, and the output of each self-attention layer can be calculated by the following formulas (3) and (4).

[0064]

[0065] Among them, head i represents the output of the i-th self-attention layer, Represents the mapping of input data Perform self-attention calculation, which can be calculated by the following formula (4).

[0066]

[0067] in, represents the embedding vector of the query, represents the embedding vector of the key, Embedding vectors representing values, which are obtained by mapping the input embeddings through a linear layer, represents the embedding dimension of the key, Represents the softmax function.

[0068] in, , indicating that the total dimension of the embedding is evenly distributed to each attention head, that is, the dimensions of the query, key, and value vectors of the i-th attention head and the key and value vectors of the j-th attention head are both , L represents the sequence length.

[0069] In operation S240, hybrid expert domain decoupling processing is performed on the sample EEG attention feature data to obtain sample EEG reconstruction data.

[0070] According to an embodiment of the present disclosure, a router can be used to adaptively select corresponding specific expert modules in the hybrid expert domain decoupling processing to process the sample EEG attention feature data based on the data features of the sample EEG attention feature data, wherein each specific expert module is composed of a feedforward layer.

[0071] In operation S250, parameters of the EEG signal representation learning model are adjusted according to the sample EEG reconstruction data and the sample EEG feature data to obtain a trained EEG signal representation learning model.

[0072] According to the embodiments of the present disclosure, the sample EEG data is encoded in the spatial domain and frequency domain so as to be integrated according to the band and channel dimensions to extract effective information of different bands and channels in the sample EEG data. The mixed expert domain decoupling processing is performed through the data features in the sample EEG attention feature data. The data features in the EEG signals can be learned in a targeted manner, and the generalization ability of the model can be enhanced, so that the model can adapt to different task requirements. Further, the learning efficiency of the model is improved, and the prediction error of the EEG model is reduced.

[0073] According to an embodiment of the present disclosure, before the spatial domain encoding of the sample EEG data is performed as mentioned above, the method also includes: obtaining original EEG signal sample data, the original EEG sample data including position information of the signal acquisition device on the target sample object; dividing the original EEG signal sample data into time periods based on a preset division time period to obtain EEG time period data; dividing the EEG time period data into bands based on a preset filtering algorithm to obtain EEG time-frequency data; obtaining EEG time-frequency-space data based on the EEG time-frequency data and the position information; and formatting the EEG time-frequency-space data to obtain sample EEG data.

[0074] According to an embodiment of the present disclosure, the above-mentioned preset filtering algorithm may be, for example, a Butterworth filtering algorithm or the like, and the present disclosure does not impose any limitation on the filtering algorithm.

[0075] According to the embodiments of the present disclosure, the sample EEG data can first be divided into non-overlapping segments of length Ts. For each segment, the EEG signal is first divided into 5 bands (respectively) by using the Butterworth filtering algorithm. The data of each band is mapped to a 3D array according to the spatial position and time sequence of the EEG clicks on the scalp, and the 3D array of each band is smoothed using Gaussian filtering or zscore. The smoothed 3D array is then stacked into a 4D form, thereby constituting the above-mentioned sample EEG data containing the time-space-frequency three-domain information.

[0076] According to the embodiments of the present disclosure, by segmenting the original EEG signal sample data based on a preset division time length to obtain EEG time period data, and further performing band division through a preset filtering algorithm, the continuous EEG signal can be divided into smaller segments to combine the data. Furthermore, by incorporating spatial dimension information into the sample EEG data, it is helpful to analyze the activities of specific brain areas. By smoothing the 3D array of each band, the noise and outliers in the data can be reduced, making the signal more stable.

[0077] According to an embodiment of the present disclosure, the above-mentioned mixed expert domain decoupling processing of the sample EEG attention feature data to obtain the sample EEG reconstruction data includes: determining at least one target-specific expert weight based on the sample EEG attention feature data and the first routing weight, wherein each target-specific expert weight represents the activation probability of the corresponding target-specific expert module, each target-specific expert module is determined from a specific expert module group based on at least one target-specific expert weight, and each target-specific expert module is used to extract specific features of the sample EEG attention feature data; determining at least one shared expert weight based on the sample EEG attention feature data and the second routing weight, wherein each shared expert module is used to extract common features of the sample attention feature data; obtaining specific expert prediction data of the sample EEG signal based on at least one target-specific expert weight and the sample EEG attention feature data; obtaining shared expert prediction data of the sample EEG signal based on at least one shared expert weight and the sample attention feature data; obtaining sample EEG reconstruction data based on the specific expert prediction data and the shared expert prediction data.

[0078] According to an embodiment of the present disclosure, the above-mentioned determining at least one target specific expert weight based on the sample EEG attention feature data and the first routing weight includes: obtaining the specific expert weight corresponding to each specific expert module in the specific expert module group based on the sample EEG attention feature data and the first routing weight; sorting the specific expert weights based on preset sorting rules to obtain sorted specific expert weights; and selecting a preset number of specific expert weights that meet the preset selection rules from the sorted specific expert weights as the target specific expert weights.

[0079] Figure 4 The structure diagram of a hybrid expert module for performing hybrid expert domain decoupling processing according to an embodiment of the present disclosure is schematically shown.

[0080] like Figure 4 As shown, the sample EEG attention feature data will be input into the specific expert module group and the shared expert module group. The specific expert module group includes multiple specific expert modules, and the shared expert module group includes multiple shared expert modules. For the specific expert module, a series of feedforward layers are used to form E experts. , for each input embedding x, router W can be used e Determine the specific expert weight for each expert , the specific expert weight can be calculated by the following formula (5).

[0081]

[0082] in, To pass through the router The obtained score vector is used to determine the activation probability of each expert, the router It can be understood as a weight matrix (i.e. the first routing weight). Map the input embedding x to get the score vector , the specific expert weight can be obtained by further calculation through the following formula (6).

[0083] (6)

[0084] Among them, p i (x) represents the specific expert weight of the i-th specific expert module, g xi Indicates g x The i-th element in |E| represents the total number of specific expert modules.

[0085] After calculating the specific expert weight of each specific expert module, the Top-K routing strategy can be used to select K specific expert modules that are most suitable for the current input embedding x from the specific expert module group, thereby obtaining specific expert prediction data. The specific expert prediction data can be obtained by the following formula (7).

[0086] (7)

[0087] Among them, SpecMoE(x) represents the specific expert prediction data, p i (x) represents the specific expert weight of the i-th specific expert module, e i (x) represents the prediction result of the i-th specific expert module.

[0088] For example, for data such as "Expert 1: 0.4, Expert 2: 0.3, Expert 3: 0.3, Expert 4: 0.08, Expert 5: 0.02", when using the Top-K routing strategy (K is set to 2), the specific expert submodules selected should be Expert 1 and Expert 2.

[0089] According to the embodiment of the present disclosure, for the shared expert module, a series of feedforward layers are also used to form F experts. , for each input embedding x, through the router Determine the shared expert weight p of each shared expert module fi (x), the shared expert weight can be calculated by the following formula (8).

[0090]

[0091] in, To pass through the router The obtained score vector is used to determine the activation probability of each expert, the router It can be understood as a weight matrix (i.e., shared expert allocation weights), the weight matrix Map the input embedding x to get the score vector , the shared expert weight p can be calculated by the method represented by formula (6) fi (x).

[0092] According to the embodiment of the present disclosure, the above-mentioned shared expert submodule group adopts a soft routing strategy, so the shared expert prediction data can be obtained by weighting the outputs of all shared experts, and the shared expert prediction data can be calculated by the following formula (9).

[0093]

[0094] Among them, ShareMoE(x) represents the shared expert prediction data, pfi(x) represents the shared expert weight of the i-th shared expert module, and fi(x) represents the prediction result of the i-th shared expert module.

[0095] Finally, based on the above-mentioned specific expert prediction data and shared expert prediction data, the output of the hybrid expert module (i.e., reconstructed data) can be obtained, which can be calculated by the following formula (10).

[0096] (10)

[0097] Among them, SSMoE(x) represents reconstructed data, SpecMoE(x) represents specific expert prediction data, and ShareMoE(x) represents shared expert prediction data.

[0098] According to the embodiments of the present disclosure, by assigning tasks to different specific expert modules for processing, different specific expert modules can focus on learning specific features or patterns in the data, thereby improving the performance of the model on specific tasks. Furthermore, by assigning weights to specific experts to determine whether to hand over the input data to corresponding specific experts for processing based on the data features of the input data, the robustness of the model can be improved, and common feature representations can be learned by sharing expert modules to enhance the generalization ability of the model.

[0099] According to an embodiment of the present disclosure, the above-mentioned parameter adjustment of the EEG signal characterization learning model according to the sample EEG reconstruction data and the sample EEG feature data to obtain a trained EEG signal characterization learning model includes: determining specific expert weights, reconstruction data and sample EEG attention feature data; determining a first allocation ratio and a second allocation ratio for allocating the sample EEG attention feature data to each specific expert module according to the specific expert weights; obtaining a first loss value according to the reconstruction data and the sample EEG attention feature data using a first loss function; obtaining a second loss value according to the first allocation ratio and the second allocation ratio using a second loss function; and adjusting the parameters of the trained EEG signal characterization learning model based on the first loss value and the second loss value to obtain a trained EEG signal characterization learning model.

[0100] According to an embodiment of the present disclosure, the first loss function may be, for example, an L1 loss function, and the second loss function may be, for example, a load balancing loss function. The following formula (11) shows a calculation formula for the L1 loss function.

[0101] (11)

[0102] Where S is the number of input embeddings, Z i is the i-th encoded EEG data segment, is the i-th reconstructed EEG data segment.

[0103] The following formula (12) shows the calculation formula of the load balancing loss function.

[0104] (12)

[0105] in, Assign to experts The embedding ratio can be calculated by the following formula (13): Assign an expert to the router i The probability ratio can be calculated by the following formula (14).

[0106] (13)

[0107] Among them, h i Indicates how many embeddings in batch B are assigned to expert e i , S represents the number of samples in batch B, that is, the number of embeddings contained in batch B.

[0108]

[0109] Where Di represents the embedding x assigned to the expert e i The probability ratio, represents the assignment of embedding x to expert e i ’s probability (i.e. the specific expert weights mentioned above).

[0110] According to an embodiment of the present disclosure, the total loss function calculation formula used in the self-supervised pre-training stage is the following formula (15).

[0111] (15)

[0112] Among them, L pretrain represents the total loss function, L1 represents the L1 loss function, L aux represents the load balancing loss function, is a hyperparameter that controls the ratio of the two loss functions.

[0113] According to an embodiment of the present disclosure, the above-mentioned EEG signal recognition method based on the domain decoupled EEG representation learning model of mixed experts may include a method for constructing an EEG signal classification model and an EEG signal classification method. Correspondingly, the EEG signal recognition device based on the domain decoupled EEG representation learning model of mixed experts may include a device for constructing an EEG signal classification model and an EEG signal classification device.

[0114] Figure 5 The flowchart of constructing an EEG signal classification model according to an embodiment of the present disclosure is schematically shown.

[0115] like Figure 5 As shown, the EEG signal classification model of this embodiment includes operations S510 to S540.

[0116] In operation S510, the second sample EEG data is input into an EEG signal representation learning module in an EEG signal classification model to obtain sample EEG signal feature representation data.

[0117] Among them, the EEG signal representation learning module is pre-loaded with an EEG signal representation learning model, and the EEG signal representation learning model is obtained through the EEG signal recognition method based on the domain decoupled EEG representation learning model of the hybrid expert.

[0118] According to an embodiment of the present disclosure, after the self-supervised pre-training stage is completed, the domain-decoupled EEG representation learning model based on the hybrid expert model has learned the characteristics of the EEG signal, which can then be combined with the MLP (Multilayer Perceptron) corresponding to a specific downstream task, and the MLP can be used as a classifier to classify the EEG signal.

[0119] In operation S520, the sample EEG signal feature representation data is input into a classification module in an EEG signal classification model to obtain a predicted classification result.

[0120] In operation S530 , a classification loss value is obtained according to the predicted classification result and the actual classification result.

[0121] According to an embodiment of the present disclosure, a cross entropy loss function may be used to calculate a loss function value to train a model. The calculation formula of the cross entropy loss function is as follows: Formula (16).

[0122] (16)

[0123] Among them, N represents the number of samples, C represents the number of sample categories, and yij represents the element in the true label, that is, the true label of the i-th sample belonging to the j-th category. Represents the probability value predicted by the model, that is, the probability that the i-th sample is predicted to be the j-th class.

[0124] In operation S540 , parameters of the EEG signal classification model are adjusted based on the classification loss value to obtain a trained EEG signal classification model.

[0125] According to an embodiment of the present disclosure, the following Table 1 shows the comparative experimental results of the EEG signal classification model according to an embodiment of the present disclosure on the DEAP (Database for Emotion Analysis using Physiological Signals) dataset.

[0126] Table 1

[0127]

[0128] It can be clearly seen that compared with EEGFuseNet (EEG fusion network), TARDGCN (Temporal AdaptiveRelational Graph Convolutional Network), TAS-Net (TemporalAttention Network), RHPRNet (Robust High-Performance EEGNetwork), EEGNet (EEG Network), TSception (TemporalSeparable Convolution Network), EEG-Conformer (EEG ConformerNetwork, EEG transformation network), LGGNet (Lightweight Graph Network), the classification accuracy of this method can be improved by at least about 1.3%.

[0129] According to an embodiment of the present disclosure, the following Table 2 shows the comparative experimental results of the EEG signal classification model according to an embodiment of the present disclosure on the BCI Competition Ⅳ-2a (the 2a data set of the 4th BCI (brain-computer interface) competition) data set, where BCI Competition Ⅳ-2a is a motor imagery recognition data set.

[0130] Table 2

[0131]

[0132] It can be obviously seen that compared with DeepCNN (Deep Convolutional Neural Network), MCNN (Multi-Channel Convolutional Neural Network), SWPC (Spectral Weighted Patch Convolutional Network), EEGNet (EEGNetwork), TSception (Temporal Separable Convolution Network), EEG-Conformer (EEG Conformer Network), and LGGNet (Lightweight Graph Network), the classification accuracy achieved by this method can be improved by at least about 2.2% compared with other algorithms.

[0133] According to an embodiment of the present disclosure, the following Table 3 shows the comparative experimental results of the EEG signal classification model according to an embodiment of the present disclosure on the STEW (Simultaneous Task EEG Workload Data Set) dataset.

[0134] Table 3

[0135]

[0136] It can be obviously seen that compared with STEW (Simultaneous Task EEG Workload), EEGNet (EEG Network), TSception (Temporal Separable Convolution Network), EEG-Conformer (EEG Conformer Network), and LGGNet (Lightweight Graph Network), the classification accuracy achieved by this method can be improved by at least about 0.6% compared with other algorithms.

[0137] According to an embodiment of the present disclosure, Table 4 shows the results of training and testing the domain decoupled EEG representation learning model based on the hybrid expert model of the present disclosure on four tasks on three public datasets DEAP, BCI Competition Ⅳ-2a and STEW, and comparing the classification results with the leave-one-subject cross-validation results without pre-training the domain decoupled EEG representation learning model based on the hybrid expert model.

[0138] Table 4

[0139]

[0140] It can be clearly seen that the classification accuracy achieved by this method can be improved by an average of 1.4% compared to the method without pre-training representation learning. This further proves that pre-training the model according to the signal recognition method of the domain decoupled EEG representation learning model based on the hybrid expert model can learn EEG representations that have both commonalities and characteristics, and enable the effectiveness of downstream tasks.

[0141] Figure 6 The flowchart of the method for classifying EEG signals according to an embodiment of the present disclosure is schematically shown.

[0142] like Figure 6 As shown, the EEG signal classification method of this embodiment includes operations S610 to S620.

[0143] In operation S610 , electroencephalogram data of a target object is acquired.

[0144] The EEG data includes the position information of the signal acquisition device on the target object.

[0145] According to the embodiments of the present disclosure, an electroencephalogram (EEG) device may be used to record the electrical activity of a target object. The location of the acquisition device is the specific placement location of the EEG electrode on the subject. The EEG signal data identifies the electrical signal from the EEG electrode, which reflects the neural activity of the brain. The EEG signal data typically contains information from multiple channels, and each channel corresponds to the location of an electrode.

[0146] In operation S620, the EEG data is input into an EEG signal classification model to obtain a classification result.

[0147] The classification result represents the intended behavior or emotional state corresponding to the EEG data of the target object, and the EEG signal classification model is obtained according to the above-mentioned method for constructing the EEG signal classification model.

[0148] For example, the classification result may indicate that the subject is imagining the movement of the left hand, right hand, feet or tongue; in an emotion recognition task, the classification result may indicate that the subject is experiencing emotions such as happiness, sadness, anger, etc.

[0149] Based on the above-mentioned EEG signal recognition method based on the domain decoupled EEG representation learning model of hybrid experts, the present disclosure also provides an EEG signal recognition device based on the domain decoupled EEG representation learning model of hybrid experts. Figure 7 The device is described in detail.

[0150] Figure 7 The structural block diagram of the EEG signal recognition device based on the domain decoupled EEG representation learning model of mixed experts according to an embodiment of the present disclosure is schematically shown.

[0151] like Figure 7 As shown, the EEG signal recognition device 700 based on the domain decoupled EEG representation learning model of the hybrid expert in this embodiment includes a spatial domain encoding module 710, a frequency domain encoding module 720, a self-attention calculation module 730, and a hybrid expert processing module 740.

[0152] The spatial domain encoding module 710 is used to perform spatial domain encoding on the sample EEG data to obtain spatial domain feature EEG data. In one embodiment, the spatial domain encoding module 710 can be used to perform the operation S210 described above, which will not be described in detail here.

[0153] The frequency domain coding module 720 is used to perform frequency domain coding on the spatial domain feature EEG data to obtain sample EEG feature data. In one embodiment, the frequency domain coding module 720 can be used to perform the operation S220 described above, which will not be described in detail here.

[0154] The self-attention calculation module 730 is used to perform multi-head self-attention calculation on the sample EEG feature data to obtain the sample EEG attention feature data. In one embodiment, the self-attention calculation module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0155] The hybrid expert processing module 740 is used to perform multi-head self-attention calculation on the sample EEG feature data to obtain the sample EEG attention feature data. In one embodiment, the hybrid expert processing module 740 can be used to perform the operation S240 described above, which will not be repeated here.

[0156] According to an embodiment of the present disclosure, the hybrid expert processing module includes: a target-specific expert weight determination submodule, which is used to determine at least one target-specific expert weight based on the sample EEG attention feature data and the first routing weight, wherein each of the target-specific expert weights represents the activation probability of the corresponding target-specific expert module, and each of the target-specific expert modules is determined from a specific expert module group based on at least one of the target-specific expert weights, and each of the target-specific expert modules is used to extract specific features of the sample EEG attention feature data; a shared expert weight determination submodule, which is used to determine at least one shared expert weight based on the sample EEG attention feature data and the second routing weight, wherein each of the shared expert modules is used to extract common features of the sample attention feature data; a specific expert prediction submodule, which is used to obtain specific expert prediction data of a sample EEG signal based on at least one of the target-specific expert weights and the sample EEG attention feature data; a shared expert prediction submodule, which is used to obtain shared expert prediction data of a sample EEG signal based on at least one of the shared expert weights and the sample attention feature data; and a reconstruction submodule, which is used to obtain the sample EEG reconstruction data based on the specific expert prediction data and the shared expert prediction data.

[0157] According to an embodiment of the present disclosure, the above-mentioned target specific expert weight determination submodule includes: a specific expert weight determination unit, which is used to obtain the specific expert weight corresponding to each specific expert module in the above-mentioned specific expert module group based on the above-mentioned sample EEG attention feature data and the above-mentioned first routing weight; a specific expert weight sorting unit, which is used to sort the above-mentioned specific expert weights based on preset sorting rules to obtain sorted specific expert weights; and a selection unit, which is used to select a preset number of specific expert weights that meet the preset selection rules from the above-mentioned sorted specific expert weights as the target specific expert weights.

[0158] According to an embodiment of the present disclosure, any multiple modules of the spatial domain coding module 710, the frequency domain coding module 720, the self-attention calculation module 730, and the hybrid expert processing module 740 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the spatial domain coding module 710, the frequency domain coding module 720, the self-attention calculation module 730, and the hybrid expert processing module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the spatial domain coding module 710, the frequency domain coding module 720, the self-attention calculation module 730, and the hybrid expert processing module 740 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0159] Figure 8 The structural block diagram of the device for forming an EEG signal classification model according to an embodiment of the present disclosure is schematically shown.

[0160] like Figure 8 As shown, the device 800 for forming an EEG signal classification model in this embodiment includes an EEG signal learning module 810 , a classification input module 820 , and a classification loss value module 830 .

[0161] The EEG signal learning module 810 is used to input the second sample EEG data into the EEG signal representation learning module in the EEG signal classification model to obtain sample EEG signal feature representation data, wherein the EEG signal representation learning module is pre-loaded with an EEG signal representation learning model, and the EEG signal representation learning model is obtained by the EEG signal recognition method based on the domain decoupled EEG representation learning model of the hybrid expert. In one embodiment, the EEG signal learning 810 can be used to perform the operation S510 described above, which will not be repeated here.

[0162] The classification input module 820 is used to input the sample EEG signal feature characterization data into the classification module in the EEG signal classification model to obtain a predicted classification result. In one embodiment, the classification input module 820 can be used to perform the operation S520 described above, which will not be repeated here.

[0163] The classification loss value module 830 is used to adjust the preset classification model based on the classification loss value to obtain a trained classification model. In one embodiment, the classification loss value module 830 can be used to perform the operation S530 described above, which will not be repeated here.

[0164] According to an embodiment of the present disclosure, any multiple modules of the EEG signal learning module 810, the classification input module 820, and the classification loss value 830 can be combined into one module for implementation, or any one of them can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the EEG signal learning module 810, the classification input module 820, and the classification loss value 830 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the EEG signal learning module 810, the classification input module 820, and the classification loss value 830 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0165] Fig. 9 The structural block diagram of the EEG signal classification device according to an embodiment of the present disclosure is schematically shown.

[0166] like Fig. 9 As shown, the EEG signal classification device 900 of this embodiment includes an EEG data acquisition module 910 and an EEG signal classification model input module 920 .

[0167] The EEG data acquisition module 910 is used to acquire EEG data of the target object, wherein the EEG data includes position information of the signal acquisition device on the target object. In one embodiment, the EEG data acquisition module 910 can be used to perform the operation S610 described above, which will not be repeated here.

[0168] The EEG signal classification model input module 920 is used to input the EEG data into the EEG signal classification model to obtain a classification result, wherein the classification result characterizes the intended behavior or emotional state corresponding to the EEG data of the target object, and the EEG signal classification model is obtained by the above-mentioned method of constructing the EEG signal classification model. In one embodiment, the EEG signal classification model input module 920 can be used to perform the operation S620 described above, which will not be repeated here.

[0169] According to an embodiment of the present disclosure, any multiple modules in the EEG data acquisition module 910 and the classification result module 920 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the EEG data acquisition module 910 and the classification result module 920 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the EEG data acquisition module 910 and the classification result module 920 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0170] Fig.10 A block diagram of an electronic device suitable for implementing an EEG signal recognition method based on a domain decoupled EEG representation learning model of a hybrid expert model according to an embodiment of the present disclosure is schematically shown.

[0171] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1009 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0172] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0173] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the input / output (I / O) interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1009 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1009 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output (I / O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1009 as needed.

[0174] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0175] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0176] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the EEG signal recognition method based on the domain decoupled EEG representation learning model of the hybrid expert model provided by the embodiment of the present disclosure.

[0177] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0178] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0179] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0180] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0181] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0182] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0183] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for EEG signal recognition based on a domain decoupled EEG representation learning model of mixed experts, characterized in that: The method comprises: Performing spatial domain encoding on the sample EEG data to obtain spatial domain feature EEG data; Performing frequency domain encoding on the spatial domain characteristic EEG data to obtain sample EEG characteristic data; Performing multi-head self-attention calculation on the sample EEG feature data to obtain sample EEG attention feature data; Performing mixed expert domain decoupling processing on the sample EEG attention feature data to obtain sample EEG reconstruction data; The parameters of the EEG signal representation learning model are adjusted according to the sample EEG reconstruction data and the sample EEG feature data to obtain a trained EEG signal representation learning model.

2. The method according to claim 1, characterized in that The step of performing mixed expert domain decoupling processing on the sample EEG attention feature data to obtain sample EEG reconstruction data includes: Determine at least one target-specific expert weight according to the sample EEG attention feature data and the first routing weight, wherein each of the target-specific expert weights represents an activation probability of a corresponding target-specific expert module, each of the target-specific expert modules is determined from a specific expert module group according to at least one of the target-specific expert weights, and each of the target-specific expert modules is used to extract specific features of the sample EEG attention feature data; Determine at least one shared expert weight according to the sample EEG attention feature data and the second routing weight, wherein each of the shared expert modules is used to extract common features of the sample attention feature data; Obtaining expert-specific prediction data of the sample EEG signal based on at least one of the target-specific expert weights and the sample EEG attention feature data; Obtaining shared expert prediction data of the sample EEG signal according to at least one of the shared expert weights and the sample attention feature data; The sample EEG reconstruction data is obtained according to the specific expert prediction data and the shared expert prediction data.

3. The method according to claim 2, characterized in that The step of determining at least one target-specific expert weight according to the sample EEG attention feature data and the first routing weight comprises: Obtaining a specific expert weight corresponding to each specific expert module in the specific expert module group according to the sample EEG attention feature data and the first routing weight; Sorting the specific expert weights based on a preset sorting rule to obtain sorted specific expert weights; A preset number of specific expert weights satisfying a preset selection rule are selected from the sorted specific expert weights as target specific expert weights.

4. The method according to claim 3, characterized in that The step of adjusting parameters of the EEG signal representation learning model according to the sample EEG reconstruction data and the sample EEG feature data to obtain a trained EEG signal representation learning model includes: Determining the specific expert weight, the reconstruction data and the sample EEG attention feature data; Determining, according to the specific expert weight, a first allocation ratio and a second allocation ratio for allocating the sample EEG attention feature data to each of the specific expert modules; Using a first loss function, according to the reconstructed data and the sample EEG attention feature data, a first loss value is obtained; Using a second loss function, according to the first allocation ratio and the second allocation ratio, a second loss value is obtained; The trained EEG signal representation learning model is parameter-adjusted based on the first loss value and the second loss value to obtain a trained EEG signal representation learning model.

5. The method according to claim 1, characterized in that Before performing spatial domain encoding on the sample EEG data, the method further includes: Acquire original EEG signal sample data, wherein the original EEG sample data includes position information of the signal acquisition device on the target sample object; Dividing the original EEG signal sample data into time periods based on a preset division time period to obtain EEG time period data; Divide the EEG time period data into bands based on a preset filtering algorithm to obtain EEG time-frequency data; Obtaining EEG time-frequency-space data based on the EEG time-frequency data and the position information; The EEG time-frequency-space data is formatted to obtain the sample EEG data.

6. The method according to claim 1, characterized in that The method further comprises: Inputting the second sample EEG data into an EEG signal characterization learning module in the EEG signal classification model to obtain sample EEG signal feature characterization data, wherein the EEG signal characterization learning module is pre-loaded with an EEG signal characterization learning model, and the EEG signal characterization learning model is obtained by the method of any one of claims 1 to 5; Inputting the sample EEG signal feature representation data into a classification module in the EEG signal classification model to obtain a predicted classification result; Obtaining a classification loss value according to the predicted classification result and the actual classification result; The parameters of the EEG signal classification model are adjusted based on the classification loss value to obtain a trained EEG signal classification model.

7. The method according to claim 1, characterized in that The method further comprises: Acquire EEG data of a target object, wherein the EEG data includes position information of a signal acquisition device on the target object; The EEG data is input into an EEG signal classification model to obtain a classification result, wherein the classification result represents the intended behavior or emotional state corresponding to the EEG data of the target object, and the EEG signal classification model is obtained according to the method of claim 6.

8. An EEG signal recognition device based on a domain decoupled EEG representation learning model of mixed experts, characterized in that: The device comprises: A spatial domain encoding module, used for performing spatial domain encoding on the sample EEG data to obtain spatial domain feature EEG data; A frequency domain coding module, used for performing frequency domain coding on the spatial domain characteristic EEG data to obtain sample EEG characteristic data; A self-attention calculation module, used to perform multi-head self-attention calculation on the sample EEG feature data to obtain sample EEG attention feature data; The hybrid expert processing module is used to perform hybrid expert domain decoupling processing on the sample EEG attention feature data to obtain sample EEG reconstruction data.

9. The device according to claim 8, characterized in that The device also includes: An EEG signal learning module, used for inputting the second sample EEG data into the EEG signal characterization learning module in the EEG signal classification model to obtain sample EEG signal feature characterization data, wherein the EEG signal characterization learning module is pre-loaded with the EEG signal characterization learning model, and the EEG signal characterization learning model is obtained by the method of any one of claims 1 to 5; A classification input module, used for inputting the sample EEG signal feature characterization data into the classification module in the EEG signal classification model to obtain a predicted classification result; The classification loss value module is used to adjust the preset classification model based on the classification loss value to obtain a trained classification model.

10. The device according to claim 8, characterized in that The device also includes: An EEG data acquisition module, used to acquire EEG data of a target object, wherein the EEG data includes position information of a signal acquisition device on the target object; An EEG signal classification model input module is used to input the EEG data into the EEG signal classification model to obtain a classification result, wherein the classification result represents the intended behavior or emotional state corresponding to the EEG data of the target object, and the EEG signal classification model is obtained according to the method of claim 6.

Images