A semantic context-dependent auditory brain-computer consciousness detection system

Abstract

The application discloses a semantic context-related auditory brain-computer interface consciousness detection system, uses a brain-computer interface and a deep learning technology to carry out consciousness detection, designs a speech context-related auditory brain-computer interface paradigm, through using animal sounds as auditory stimulation, adds a semantic context related to the stimulation to improve the attention of a subject to the paradigm, thereby improving the quality of an event-related potential induced by the paradigm and improving the performance of the brain-computer interface. Through improvement of an EEG-Inception model for feature extraction and classification, the EEG-Inception model can extract more effective electroencephalogram signal features and also has a better classification effect under a small sample size. Through innovation and improvement of the brain-computer interface paradigm and a detection and classification algorithm, the application improves the performance of the auditory brain-computer interface and solves the problem that some patients with consciousness disorders cannot use a visual brain-computer interface for consciousness detection because they cannot control eye movement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of auditory brain-computer interface consciousness detection, and particularly to an auditory brain-computer consciousness detection system related to semantic context. BACKGROUND

[0002] Brain-computer interface is a technology that interacts with the outside world through electrophysiological phenomena. It realizes interaction by acquiring and analyzing the brain electrical signals of users. This communication method is widely used in the treatment and diagnosis of patients with consciousness disorders who cannot normally communicate with the outside world. Consciousness detection of brain-computer interface needs to induce event-related potentials of subjects through stimulation, and to determine whether the subjects react to the specified stimulation according to the instructions by detecting the characteristic brain electrical signals. At present, deep learning is often used to decode brain electrical signals in brain-computer interface. Among them, convolutional neural network can extract brain electrical features corresponding to different frequency bands by controlling the size of convolution kernel, and it is a commonly used deep learning network for event-related potential detection at present.

[0003] Compared with auditory brain-computer interface, visual brain-computer interface has higher accuracy and information transmission rate. However, visual brain-computer interface depends on the visual control ability of subjects, so it cannot be applied to patients with impaired vision. In order to improve the universality of brain-computer interface and obtain more accurate results in patients with impaired vision, it is necessary to improve the performance of auditory brain-computer interface from two aspects of brain-computer interface paradigm and event-related potential detection algorithm, so as to apply it to consciousness detection of most patients with consciousness disorders. From the perspective of human physiology, the auditory system is more sensitive to natural sounds and drives auditory neurons to be in the best state of information transmission. At the same time, improving the attention of subjects during detection can also effectively improve the performance of brain-computer interface paradigm. In terms of detection algorithm, EEG-Incption is a commonly used model for event-related potential detection. It extracts brain electrical features of different time window sizes through parallel multi-scale convolution kernels, so as to extract brain electrical signal features of different frequency bands. However, due to its complex structure, it often needs large training data to train the model. At the same time, brain electrical signals need to collect signals of brain electrical channels at different positions of the brain. Selecting effective brain electrical channels can make the model extract more effective brain electrical features, thereby improving the classification accuracy of the model and the performance of brain-computer interface. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings and deficiencies of the prior art and provide a semantic context-related auditory brain-computer consciousness detection system, which can induce stronger event-related potentials by improving the attention of the subject to the auditory stimuli in the paradigm, and add an ECA channel attention mechanism module to the model and use a prototype network to classify the samples, thereby improving the classification accuracy under small sample conditions.

[0005] To achieve the above-mentioned purpose, the technical scheme provided by the present application is as follows: a semantic context-related auditory brain-computer consciousness detection system, comprising:

[0006] The event-related potential evoking and collecting module presents an auditory brain-computer interface paradigm to the subject and collects the electroencephalogram data of the subject during the paradigm; during the paradigm, the subject needs to pay attention to the target stimuli in a series of stimuli to evoke event-related potentials, and a semantic context related to the auditory stimuli will appear during the paradigm to improve the attention of the subject to the paradigm, thereby inducing more significant event-related potentials and improving the subsequent classification accuracy;

[0007] The electroencephalogram feature extraction and classification module inputs the electroencephalogram data collected by the event-related potential evoking and collecting module into the improved EEG-Inception model to extract features and classify; wherein the improvements of the improved EEG-Inception model include: ①adding an ECA channel attention mechanism module for weight distribution of the electroencephalogram channels to facilitate the model to extract more effective electroencephalogram features; ②replacing the original classifier of the EEG-Inception model with a prototype network, which classifies data samples by calculating the distance between the feature mapping of each category and the target prototype in the feature space, thereby reducing the amount of data required for training the model;

[0008] The result determination and output module divides the electroencephalogram data processed by the electroencephalogram feature extraction and classification module into two categories: target stimuli and non-target stimuli, determines the correctness of the test by comparing whether the target stimuli of the two are consistent, and gives the subject audio feedback of correct or incorrect, and finally evaluates the consciousness level of the subject according to the overall accuracy of all tests.

[0009] Further, the event-related potential evoking and collecting module specifically performs the following operations:

[0010] In each round of the auditory brain-computer interface paradigm, the stimuli will be divided into target stimuli and non-target stimuli; in the prompt stage before the test begins, the subject will be told the target stimuli to pay attention to in this round of test; during the test, the event-related potentials of the subject will be induced using the calls of two animals as stimuli; in each round of the test, one of the two animal calls will be randomly selected as the target stimulus, and the other as the non-target stimulus; before each animal call stimulus appears, a sentence describing the animal will be used as the semantic context; in each round of the test, all the sentences constitute a complete and meaningful story to improve the subject's attention to the paradigm, at the same time, the sentences as interference stimuli reduce the probability of occurrence of the target stimulus, thereby inducing a larger amplitude of event-related potentials;

[0011] In the collection of electroencephalogram data, the subject wears an electroencephalogram cap connected to an electroencephalogram signal amplifier to collect the electroencephalogram data of the subject during the paradigm and transmit it to the computer; the computer will store the received electroencephalogram data according to the rounds of the test, and perform the following preprocessing: ① The electroencephalogram data is classified according to the animal call corresponding to the stimulus and labeled; ② The electroencephalogram data is cut according to the starting of the stimulus to a preset time length; ③ The electroencephalogram data is filtered in the range of 0.1Hz to 60Hz; ④ The electroencephalogram data is down-sampled to 128Hz; ⑤ The electroencephalogram data is z-score standardized, and the z-score standardization formula is as follows:

[0012]

[0013] In the formula, Z is the standardized electroencephalogram data, X is the single electroencephalogram data after step ④ processing, is the average of all electroencephalogram data, and s is the standard deviation of all electroencephalogram data; the preprocessed electroencephalogram data format is: according to the animal call, the stimulus is divided into two categories, each category contains n samples, and each sample is a three-dimensional matrix with the size of electroencephalogram channel number x sampling number x 1; finally, the three-dimensional matrix is input into the electroencephalogram feature extraction and classification module.

[0014] Further, the electroencephalogram feature extraction and classification module classifies the input two categories of samples into target stimuli and non-target stimuli for subsequent test result discrimination, which extracts features and classifies the input by improving the EEG-Inception model, and the specific steps are as follows:

[0015] a、The input first passes through the ECA channel attention mechanism module to assign weights to each electroencephalogram channel, and the weight calculation formula of each channel is as follows:

[0016]

[0017] In the formula, ω cis the weight of the cth channel, σ is the Sigmoid activation function, C is the total number of electroencephalogram channels, w c is the learning parameter corresponding to the adjacent c'th channel, is the set of k adjacent channels of the cth channel, is the adjacent c'th channel of the cth channel; in the ECA channel attention mechanism module, first, the global average pooling operation is performed on the input three-dimensional matrix to reduce the dimension, then the local features between the channels and the adjacent channels are captured through one-dimensional convolution, and then the Sigmoid activation function is used to obtain the channel weight corresponding to each channel, and finally each channel is multiplied by the corresponding weight obtained by the above formula to obtain a feature matrix with the same size as the input after the channel weight distribution, so as to realize the electroencephalogram channel screening and extract more effective features;

[0018] b, after the ECA channel attention mechanism module, the feature matrix will pass through a parallel Inception module, which uses three different size convolution kernels to extract different frequency band electroencephalogram features; in the Inception module, the input feature matrix is first passed through a parallel convolution layer composed of three convolution kernels with sizes of 64x1, 32x1 and 16x1, which extracts 2Hz, 4Hz and 8Hz frequency band features; After each convolution kernel, a 1x8 depth convolution is performed to reduce the dimension of the obtained features; Finally, the features obtained by the three parallel convolution layers are merged through the connection layer to obtain a 128x1x48 feature output;

[0019] c, the features extracted by the Inception module are reduced in dimension by pooling and extracted into higher dimension features, including a 4x1 pooling layer for average pooling operation to reduce the dimension of the obtained features, an 8x1 depth convolution layer to extract higher dimension features, and a 2x1 pooling layer for average pooling to further reduce the dimension of the features, to obtain the feature matrix of each sample;

[0020] d, calculate the feature map of all samples of different categories, and calculate the cosine distance between the feature map and the prototype of the positive sample, the class with small cosine distance is classified as the target stimulus, and the class with large cosine distance is classified as the non-target stimulus, the specific operation is as follows:

[0021] First, the feature matrix obtained by step ③ of all samples of different categories is averaged to obtain the feature map of each category:

[0022]

[0023] In the formula, V i represents the feature map of the ith category, f ijis the feature matrix of the jth sample of the ith category, and n is the total number of samples in each category;

[0024] The target stimulus is taken as a positive sample, and the non-target stimulus is taken as a negative sample. The positive sample feature mapping calculated by the feature matrix of the target stimulus is taken as a positive sample prototype of the model, which is used to calculate the cosine distance of the feature mapping of each category in the validation set. The calculation formula of the cosine distance is as follows:

[0025]

[0026] In the formula, S i represents the cosine distance of the feature mapping of the ith category and the positive sample prototype, V p represents the positive sample prototype. The smaller the cosine distance value is, the more similar the feature mapping of the category is to the positive sample prototype in the feature space. Therefore, the category with the smallest cosine distance is taken as the positive sample category, and the others are taken as the negative sample category. The positive sample category corresponds to the target stimulus, and is represented by label 1. The negative sample category corresponds to the non-target stimulus, and is represented by label -1.

[0027] Further, the calculation formula of the loss function of the improved EEG-Inception model is as follows:

[0028] l i = L(S i ,y i )

[0029] In the formula, l i is the loss value of the ith category, the mean square error is selected as the loss function L, and y i is the label of the ith category. In the training process, the model is continuously iteratively trained through back propagation, and the model that minimizes the loss value of the validation set is taken as the optimal model for subsequent detection and classification of the test set.

[0030] Further, the result determination and output module specifically performs the following operations:

[0031] The EEG data processed by the EEG feature extraction and classification module is divided into positive samples and negative samples, which correspond to target stimuli and non-target stimuli, respectively. The test result is obtained by comparing whether the target stimulus informed to the subject in the comparison prompt stage is consistent with the classification result. If the test is correct, applause and voice will be played to inform the subject that the answer is correct. If it is wrong, the subject will be informed by voice that the answer is wrong. After detection by a plurality of test groups containing a plurality of tests, the result determination and output module evaluates the consciousness level of the subject according to the overall accuracy of all tests. If the overall accuracy of the subject is higher than the preset percentage of the random level, it is considered that the subject has consciousness, otherwise not. The result is displayed on the screen.

[0032] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0033] 1. The present application designs a speech context-related auditory brain-computer interface paradigm, which can improve the attention of the subject to the auditory stimulus in the paradigm, thereby inducing more significant event-related potentials, thereby obtaining higher classification accuracy.

[0034] 2. The present application introduces an ECA channel attention mechanism module, so that the model can assign weights to different electroencephalogram channels, thereby obtaining more effective electroencephalogram features.

[0035] 3. The present application uses a prototype network to replace the original classifier of the EEG-Inception model, which improves the classification accuracy of the model in a small sample scenario by mapping the features to a feature space. BRIEF DESCRIPTION OF DRAWINGS

[0036] Fig. 1 The present application is designed for an auditory brain-computer interface paradigm flowchart.

[0037] Fig. 2 The principle diagram for improving the EEG-Inception model.

[0038] Fig. 3 The present application is designed for an auditory brain-computer interface paradigm flowchart. DETAILED DESCRIPTION

[0039] The present application will be further described below in conjunction with specific embodiments.

[0040] The present embodiment discloses a semantic context-related auditory brain-computer consciousness detection system, comprising:

[0041] Event-related potential induction and acquisition module: present an auditory brain-computer interface paradigm to the subject, and acquire the electroencephalogram data of the subject during the paradigm process; during the paradigm process, the subject needs to pay attention to the target stimulus in a series of stimuli to induce event-related potentials, and a semantic context related to the auditory stimulus will appear during the paradigm process, which is used to improve the attention of the subject to the paradigm, thereby inducing more significant event-related potentials, and thereby improving the subsequent classification accuracy;

[0042] The EEG feature extraction and classification module inputs the EEG data collected by the event-related potential induction and collection module into the trained improved EEG-Inception model for feature extraction and classification. The improvements of the improved EEG-Inception model include: (1) adding an ECA channel attention mechanism module for weight distribution of EEG channels to facilitate the model to extract more effective EEG features; and (2) replacing the original classifier of the EEG-Inception model with a prototype network, which classifies data samples by calculating the distance between the feature mapping of each category and the target prototype in the feature space, thereby reducing the amount of data required for training the model.

[0043] The result determination and output module determines the correctness of the test by comparing whether the target stimuli of the prompt stage and the classification results are consistent, and gives the subject audio feedback of correct or incorrect, and finally evaluates the consciousness level of the subject according to the overall accuracy of all tests.

[0044] Specifically, the event-related potential induction and collection module specifically performs the following operations:

[0045] In each round of testing of the auditory brain-computer interface paradigm, the stimuli will be divided into target stimuli and non-target stimuli. In the prompt stage before the test starts, the subject will be told the target stimulus that needs to be paid attention to in this round of test. During the test, the event-related potentials of the subject are induced using the calls of two animals as stimuli. In each round of testing, one of the two animal calls is randomly selected as the target stimulus, and the other is selected as the non-target stimulus. Before each animal call stimulus appears, a sentence describing the animal will be used as a semantic background. In each round of testing, all the sentences form a complete and meaningful story to improve the subject's attention to the paradigm. At the same time, the sentences as interference stimuli reduce the probability of occurrence of the target stimulus, thereby inducing a larger amplitude of event-related potentials.

[0046] In the collection of EEG data, the EEG data of the subject during the paradigm is collected by wearing an EEG cap connected to an EEG signal amplifier on the subject and transmitted to the computer. The computer will store the received EEG data according to the rounds of testing and perform the following preprocessing: (1) the EEG data is classified according to the animal call corresponding to the stimulus and labeled; (2) the EEG data is cut according to the starting of the stimulus to a pre-set time length; (3) the EEG data is filtered in the range of 0.1 Hz to 60 Hz; (4) the EEG data is down-sampled to 128 Hz; (5) the EEG data is subjected to z-score standardization processing, and the z-score standardization formula is as follows:

[0047]

[0048] In the formula, Z is the normalized electroencephalogram data, X is the single electroencephalogram data processed by step ④, is the average of all electroencephalogram data, s is the standard deviation of all electroencephalogram data; the format of the preprocessed electroencephalogram data is: according to the animal call, the stimulation is divided into two categories, each category contains n samples, and each sample is a three-dimensional matrix with the size of electroencephalogram channel number x sampling number x 1; finally, the three-dimensional matrix is input into the electroencephalogram feature extraction and classification module.

[0049] Specifically, the electroencephalogram feature extraction and classification module classifies the input two categories of samples into target stimulation and non-target stimulation, which is used for subsequent test result discrimination, and it extracts and classifies the input features by improving the EEG-Inception model, and the specific steps are as follows:

[0050] a、The input is first weighted and distributed to each electroencephalogram channel by the ECA channel attention mechanism module, and the weight calculation formula of each channel is as follows:

[0051]

[0052] In the formula, ω c is the weight of the cth channel, σ is the Sigmoid activation function, C is the total number of electroencephalogram channels, w c is the learning parameter corresponding to the adjacent c'th channel, is the set of k adjacent channels of the cth channel, is the c'th channel adjacent to the cth channel; in the ECA channel attention mechanism module, the input three-dimensional matrix is first subjected to global average pooling operation for dimension reduction, then one-dimensional convolution is used to capture the local features between the channels and the adjacent channels, and then the Sigmoid activation function is used to obtain the channel weight corresponding to each channel, finally each channel is multiplied by the corresponding weight obtained by the above formula, to obtain a feature matrix with the same size as the input and subjected to channel weight distribution, so as to realize electroencephalogram channel screening and extract more effective features;

[0053] b、After passing through the ECA channel attention mechanism module, the feature matrix will pass through an Inception module in parallel, which uses three different size convolution kernels to extract different frequency band electroencephalogram features; in the Inception module, the input feature matrix is first passed through a parallel convolution layer composed of three convolution kernels with sizes of 64x1, 32x1 and 16x1 in parallel to extract features in 2Hz, 4Hz and 8Hz frequency bands; a depth convolution of 1x8 is performed after each convolution kernel to reduce the dimension of the obtained features; finally, the features obtained by the three parallel convolution layers are merged through a connection layer to obtain a 128x1x48 feature output;

[0054] c、The features extracted through the Inception module are reduced in dimension through pooling and deep convolution to extract higher-dimensional features, including a 4x1 pooling layer for average pooling operation to reduce the dimension of the obtained features, an 8x1 deep convolution layer to extract higher-dimensional features, and a 2x1 pooling layer for average pooling to further reduce the dimension of the features, obtaining the feature matrix of each sample;

[0055] d、Calculate the feature map of all samples of different categories, and calculate the cosine distance between the feature map and the positive sample prototype. The category with small cosine distance is classified as target stimulus, and the category with large cosine distance is classified as non-target stimulus. The specific operation is as follows:

[0056] First, the feature matrix obtained in step ③ for all samples of different categories is averaged to obtain the feature map of each category:

[0057]

[0058] In the formula, V i represents the feature map of the i-th category, f ij is the feature matrix of the j-th sample of the i-th category, and n is the total number of samples in each category;

[0059] The target stimulus is taken as the positive sample, and the non-target stimulus is taken as the negative sample. The feature map of the positive sample calculated by the feature matrix of the target stimulus is taken as the positive sample prototype of the model, which is used to calculate and verify the cosine distance between the feature map of each category in the validation set. The formula for calculating the cosine distance is as follows:

[0060]

[0061] In the formula, S i represents the cosine distance between the feature map of the i-th category and the positive sample prototype, and V pThe cosine distance represents the positive sample prototype; the smaller the cosine distance value, the more similar the feature mapping of this category is to the positive sample prototype in the feature space; therefore, the category with the smallest cosine distance is taken as the positive sample category, and the others are negative sample categories; the positive sample category corresponds to the target stimulus, denoted by label 1; the negative sample category corresponds to the non-target stimulus, denoted by label -1.

[0062] Specifically, the formula for calculating the loss function of the improved EEG-Inception model is as follows:

[0063] l i =L(S) i ,y i )

[0064] In the formula, l i For the loss value of the i-th category, the mean squared error is chosen as the loss function L, y i Let be the label of the i-th category; during training, the model is iteratively trained through backpropagation to obtain the model that minimizes the loss value on the validation set as the optimal model, which is then used for detection and classification on the subsequent test set.

[0065] Specifically, the result determination and output module performs the following operations:

[0066] The EEG data extracted and classified by the EEG feature extraction and classification module will be divided into positive samples and negative samples, corresponding to target stimuli and non-target stimuli, respectively. The test result is obtained by comparing whether the target stimulus told to the subject in the comparison prompt stage is consistent with the target stimulus in the classification result. If the test is correct, applause and voice will be played to inform the subject that the answer is correct; if it is incorrect, the subject will be informed of the incorrect answer through voice. After testing multiple test groups containing several tests, the result judgment and output module will evaluate the subject's level of consciousness based on the overall accuracy of all tests. If the subject's overall accuracy is higher than 63% at random level, the subject is considered to be conscious; otherwise, the subject is considered not conscious, and the result will be displayed on the screen.

[0067] like Figs. 1-3 As shown, the specific implementation of the semantic background-related auditory brain-computer interface consciousness detection system in this embodiment includes the following steps:

[0068] 1) Device Fitting: The subject should be fitted with a 34-channel EEG cap conforming to the international 10-20 system standard, along with an in-ear headset for cues and auditory stimulation. After the data acquisition device is fitted, the test should only begin after all channel impedances have dropped below 5kΩ.

[0069] 2) Paradigm procedure: Each detection of each subject needs to complete two or more groups of tests, each group contains 16 tests. Among them, the first group is used to collect training data to train the model, and the rest is used to collect test data for evaluation. As shown in Fig. 1 , it is the flowchart of the paradigm 1 test of this embodiment. In the test, the call of a chicken or a dog is used as an auditory stimulus to induce the event-related potential of the subject. Each round of test will randomly select one of the two animal calls (probability of 50%) as the target stimulus, and the other as the non-target stimulus. In the prompt stage before the test starts, the subject is informed by the prompt that the target stimulus needs to be counted silently. Before each animal call stimulus appears, there will be a sentence describing the animal as a semantic background. The background of each round of test has a total of 8 sentences, each corresponding to one of the chicken or dog (4 for each). For example: "In the afternoon, a dog is taking a nap in the shade." Or "In the morning, the chicken in the yard is foraging." After each sentence, the corresponding animal call stimulus will appear. All the sentences form a complete and meaningful story to improve the subject's attention to the paradigm. At the same time, the sentence as a distractor stimulus reduces the probability of occurrence of the target stimulus, thereby inducing a larger amplitude of event-related potential.

[0070] 3) Data preprocessing: The EEG data of the subject during the paradigm is collected by wearing an EEG cap connected to an EEG signal amplifier on the subject and transmitted to the computer. The computer will store the received EEG data according to the test round and perform the following preprocessing: ① The EEG data is classified according to the animal call corresponding to the stimulus and labeled; ② The EEG data is truncated according to the length from the stimulus to 1000ms after the start; ③ The EEG data is filtered in the range of 0.1Hz to 60Hz; ④ The EEG data is downsampled to 128Hz; ⑤ The EEG data is standardized by z-score.

[0071] 4) Feature extraction and classification: As shown in Fig. 2 , it is an improved EEG-Inception model for event-related potential classification, which needs to classify the input two-class samples into target stimulus and non-target stimulus according to the detected event-related potential. The model is divided into feature extraction and classification according to the function. The EEG data obtained through the data preprocessing step is first input into the feature extraction part to extract and reduce the dimension of the EEG features, and then the obtained feature matrix is input into the classification part to classify the two-class samples into target and non-target.

[0072] In the feature extraction part, the input is first passed through the ECA channel attention mechanism module to assign weights to each EEG channel, select the more important EEG channels for subsequent classification, and extract the more effective EEG channels for classification. In the ECA channel attention mechanism module, the input three-dimensional matrix is first subjected to global average pooling operation for dimension reduction, then one-dimensional convolution is used to capture the local features between channels and adjacent channels, and then Sigmoid activation function is used to obtain the channel weight corresponding to each channel. The weight represents the importance of each channel in the overall feature. Finally, the calculated channel weight is used to weight the calculation of each channel of the original input EEG.

[0073] After passing through the ECA channel attention mechanism module, the feature matrix will pass through a parallel Inception module, which uses three different size convolution kernels to extract EEG features of different frequency bands. In the Inception module, the input feature matrix is first passed through a parallel convolution layer composed of three convolution kernels with sizes of 64x1, 32x1 and 16x1. The three convolution kernels correspond to time windows of 500ms, 250ms and 125ms respectively, which are used to extract EEG features of 2Hz, 4Hz and 8Hz frequency bands. After convolution operation by each convolution kernel, a 1x8 deep convolution layer is used to reduce the dimension of the obtained features, reducing the number of parameters required for training. Then the features obtained by the three parallel convolution layers are merged through a connection layer to obtain a 128x1x48 feature output. After multi-scale feature extraction by the Inception module, the obtained features are reduced by pooling and deep convolution to extract higher dimensional features. This includes a 4x1 pooling layer for average pooling operation to reduce the dimension of the obtained features, an 8x1 deep convolution layer to extract higher dimensional features, and a 2x1 pooling layer for average pooling to further reduce the dimension of the features, obtaining the feature matrix of each sample.

[0074] After obtaining the feature matrix, the feature matrix of all samples of different categories is averaged to obtain the feature map of each category.

[0075] In the model training stage, the training data is divided into a training set and a validation set, with a ratio of 3:1. The training set is used to train the model parameters and obtain the positive sample prototype of the model. The target stimulus in the training set is used as the positive sample, and the non-target stimulus is used as the negative sample. The positive sample feature mapping calculated from the feature matrix of the positive sample will be used as the positive sample prototype of the model. After obtaining the feature mapping of each class in the validation set and the test set, the cosine distance between the feature mapping of each class and the positive sample prototype is calculated. The smaller the cosine distance value, the more similar the feature mapping of the class is to the positive sample prototype in the feature space. Therefore, the class with the smallest cosine distance is classified as the positive sample class, and the others are classified as the negative sample class. The positive sample class corresponds to the target stimulus, and is represented by label 1. The negative sample class corresponds to the non-target stimulus, and is represented by label -1. The cosine distance between the feature mapping of each class in the validation set and the positive sample prototype is calculated, and the loss value is obtained by combining the actual label of the sample. Each training set is iteratively trained for 200 times, and the model with the smallest validation set loss value and its positive sample prototype are selected and saved for subsequent testing. In the testing stage, the optimal model trained by the subject is used to predict the electroencephalogram data collected by the subject in the testing stage. The test set data is put into the feature extraction module for feature extraction operation, and then the feature mapping of different classes is calculated according to the above method, and the cosine distance between the optimal model corresponding to the positive sample prototype is calculated. The class with the smallest cosine distance is classified as the target stimulus, and the rest is classified as the non-target stimulus.

[0076] 5) Result judgment and output: In the testing stage, the electroencephalogram data processed by the electroencephalogram feature extraction and classification module will be divided into positive samples and negative samples, corresponding to target stimuli and non-target stimuli respectively; the test result is obtained by comparing whether the target stimulus informed to the subject in the comparison prompt stage is consistent with the target stimulus of the classification result; if the test is correct, applause and voice will be played to inform the subject that the answer is correct. If it is wrong, the subject will be informed by voice that the answer is wrong. After a group of test groups, the test results of all the questions in this group and the overall accuracy of this group are output. After detecting a plurality of test groups containing several tests, the result judgment and output module will evaluate the consciousness level of the subject according to the overall accuracy of all tests, and display the consciousness level of the subject on the screen.

[0077] The above-described embodiments are only the preferred embodiments of the present application, and do not limit the scope of the present application. Any changes made in shape or principle according to the present application should be covered within the scope of protection of the present application.

Claims

1. A semantic context-dependent auditory brain-machine consciousness detection system, characterized in that, Comprise: Event-related potential induction and acquisition module: present an auditory brain-computer interface paradigm to the subject, and acquire the electroencephalogram data of the subject during the paradigm; During the paradigm, the subject needs to pay attention to the target stimulus in a series of stimuli to induce event-related potentials, and a semantic context related to the auditory stimulus will appear during the paradigm to improve the subject's attention to the paradigm, thereby inducing more significant event-related potentials, and thus improving the subsequent classification accuracy; EEG feature extraction and classification module: input the EEG data collected by the event-related potential induction and acquisition module into the improved EEG-Inception model to extract features and classify; wherein the improvements of the improved EEG-Inception model include: ①adding an ECA channel attention mechanism module for weight distribution of EEG channels to facilitate the model to extract more effective EEG features; ②using a prototype network to replace the original classifier of the EEG-Inception model, which classifies data samples by calculating the distance between the feature mapping of each class and the target prototype in the feature space, thereby reducing the amount of data required for training the model; Result determination and output module: the EEG data processed by the EEG feature extraction and classification module will be classified into target stimulus and non-target stimulus, and the correctness of the test will be determined by comparing whether the target stimulus of the prompt stage and the classification result are consistent, and the subject will be given audio feedback of correct or incorrect, and finally the consciousness level of the subject will be evaluated according to the overall accuracy of all tests.

2. The semantic context-dependent auditory brain-computer consciousness detection system according to claim 1, wherein, The event-related potential induction and acquisition module specifically performs the following operations: In each round of testing of the auditory brain-computer interface paradigm, the stimulus will be divided into target stimulus and non-target stimulus; in the prompt stage before the test, the subject will be told the target stimulus that needs to be paid attention to in this round of test; during the test, the calls of two animals are used as stimuli to induce the event-related potentials of the subject, and one of the two animal calls will be selected as the target stimulus and the other as the non-target stimulus in each round of test; before each animal call stimulus appears, a sentence describing the animal will appear as a semantic context; in each round of test, all sentences form a complete and meaningful story to improve the subject's attention to the paradigm, at the same time, the sentence as a interference stimulus reduces the probability of occurrence of the target stimulus, thereby inducing a larger amplitude of event-related potentials; In the collection of electroencephalogram data, the electroencephalogram data of the subject during the paradigm is collected by wearing an electroencephalogram cap connected with an electroencephalogram signal amplifier on the subject and transmitted to the computer; the computer will store the received electroencephalogram data according to the test rounds, and the following pretreatment will be carried out: ① The electroencephalogram data is classified according to the animal call corresponding to the stimulus and is labeled; ② The electroencephalogram data is intercepted according to the preset time length from the start of the stimulus; ③ The electroencephalogram data is filtered in the range of 0.1Hz to 60Hz; ④ The electroencephalogram data is down-sampled to 128Hz; ⑤ The electroencephalogram data is subjected to z-score standardization processing, and the z-score standardization formula is as follows: In the formula, Z is the normalized electroencephalogram data, X is the single electroencephalogram data processed by step 4, is the average of all electroencephalogram data, s is the standard deviation of all electroencephalogram data; the format of the preprocessed electroencephalogram data is: according to the animal call, the stimulation is divided into two categories, each category contains n samples, and each sample is a three-dimensional matrix with the size of electroencephalogram channel number * sampling number * 1; finally, the three-dimensional matrix is input into the electroencephalogram feature extraction and classification module.

3. The semantic context-dependent auditory brain-computer consciousness detection system according to claim 2, wherein, The EEG feature extraction and classification module is to classify the input two types of samples into target stimulus and non-target stimulus, which is used for subsequent test result discrimination. The EEG feature extraction and classification module improves the EEG-Inception model to perform feature extraction and classification on the input, and the specific steps are as follows: a. The input first passes through the ECA channel attention mechanism module to assign weights to each electroencephalogram channel. The weight calculation formula of each channel is as follows: In the formula, ω c is the weight of the cth channel, σ is the Sigmoid activation function, C is the total number of electroencephalogram channels, w c is the learning parameter corresponding to the adjacent c'th channel, is the set of k adjacent channels of the cth channel, is the c'th channel adjacent to the cth channel; in the ECA channel attention mechanism module, first, the global average pooling operation is performed on the input three-dimensional matrix to reduce the dimension, then one-dimensional convolution is used to capture the local features between the channels and the adjacent channels, and then the Sigmoid activation function is used to obtain the channel weight corresponding to each channel, and finally each channel is multiplied by the corresponding weight obtained by the above formula to obtain a feature matrix with the same size as the input, which has been subjected to channel weight distribution, so as to realize the electroencephalogram channel screening and extract more effective features; b. After passing through the ECA channel attention mechanism module, the feature matrix will pass through a parallel Inception module, which uses three different size convolution kernels to extract electroencephalogram features of different frequency bands. In the Inception module, the input feature matrix first passes through a parallel convolution layer composed of three convolution kernels with sizes of 64x1, 32x1 and 16x1, which extracts features of 2Hz, 4Hz and 8Hz frequency bands from the feature matrix. After each convolution kernel, a 1x8 depth convolution is performed to reduce the dimension of the obtained features. Finally, the features obtained from the three parallel convolution layers are combined through a connection layer to obtain a 128x1x48 feature output; c. The features extracted by the Inception module are reduced in dimension by pooling and deep convolution to extract higher-dimensional features, including a 4x1 pooling layer for average pooling to reduce the dimension of the obtained features, an 8x1 deep convolution layer to extract higher-dimensional features, and a 2x1 pooling layer for average pooling to further reduce the dimension of the features, obtaining the feature matrix of each sample; d. Calculate the feature mapping of all samples of different categories, and calculate the cosine distance between the feature mapping and the positive sample prototype. The category with smaller cosine distance is classified as target stimulus, and the category with larger cosine distance is classified as non-target stimulus. The specific operation is as follows: First, the feature matrix of each category is obtained by averaging the feature matrix of all samples of different categories obtained in step ③: In the formula, V i represents the feature mapping of the i-th category, f ij is the feature matrix of the j-th sample of the i-th category, and n is the total number of samples of each category. The target stimulus is taken as the positive sample, and the non-target stimulus is taken as the negative sample. The positive sample feature mapping calculated by the feature matrix of the target stimulus is taken as the positive sample prototype of the model, which is used to calculate and verify the cosine distance of each category feature mapping. The cosine distance calculation formula is as follows: where S i represents the cosine distance between the feature mapping of the i-th class and the positive sample prototype, V p represents the positive sample prototype; the smaller the cosine distance value, the more similar the feature mapping of the class is to the positive sample prototype in the feature space; therefore, the class with the smallest cosine distance is taken as the positive sample class, and the others are negative sample classes; the positive sample class corresponds to the target stimulus, and is represented by label 1; the negative sample class corresponds to the non-target stimulus, and is represented by label -1.

4. The semantic context-dependent auditory brain-computer consciousness detection system according to claim 3, wherein, The loss function calculation formula of the improved EEG-Inception model is as follows: l i = L(S i ,y i ) In the formula, l i is the loss value of the i-th category, and the mean square error is selected as the loss function L, y i is the label of the i-th category; In the training process, the model is iteratively trained through back propagation, and the model that minimizes the loss value of the validation set is obtained as the optimal model for subsequent detection and classification of the test set.

5. The semantic context-dependent auditory brain-computer consciousness detection system according to claim 4, wherein, The result determining and outputting module specifically performs the following operations: The EEG data processed by the EEG feature extraction and classification module is divided into positive samples and negative samples, corresponding to target stimuli and non-target stimuli respectively; the test result is obtained by comparing whether the target stimuli informed to the subject in the comparison prompt stage are consistent with the target stimuli in the classification result; if the test is correct, applause and voice will be played to inform the subject that the answer is correct; if it is wrong, the subject will be informed by voice that the answer is wrong; after detection by a plurality of test groups containing a plurality of tests, the result determining and outputting module evaluates the consciousness level of the subject according to the overall accuracy of all tests; if the overall accuracy of the subject is higher than the preset percentage of the random level, the subject is considered to have consciousness, otherwise not, and the result is displayed on the screen.

Images