Automatic detection and attribution analysis method based on cognitive state of EEG (electroencephalogram) learner

Abstract

The invention discloses an automatic detection and attribution analysis method based on the cognitive state of an EEG learner, and the method comprises the following steps: S1, experiment design: designing high-order and low-order learning activities, recruiting a subject, collecting EEG signals when the subject participates in the learning activities, obtaining EEG data, and constructing LADataset; s2, cognitive state detection: based on the constructed learning activity data, respectively detecting high-order and low-order cognitive states of the learner by adopting CNN, LSTM and CNN-LSTM models; and S3, attribution analysis: aiming at the collected EEG data, regarding each channel as a to-be-explained feature, then utilizing a linear regression model to fit a local boundary of an explaining model to obtain a contribution value of each channel of a to-be-explained sample, and finally, selecting a prediction sample with relatively high confidence coefficient, and calculating an average value of explaining results of the prediction sample. The accuracy rate of the method reaches 99%, and the effectiveness of the method is verified.

Description

Technical Field

[0001] The present invention relates to the field of attribution analysis, and in particular to an automatic detection and attribution analysis method for learners' cognitive states based on EEG. Background Art

[0002] Detecting the high and low cognitive states of learners during learning activities helps to evaluate and improve learners' cognitive abilities, and is an important part of the development of higher-order thinking abilities. The use of EEG (electroencephalogram) signals combined with deep learning algorithms effectively realizes automatic detection of cognitive states. However, we cannot obtain EEG features that distinguish cognitive states from such methods, and thus cannot understand the neural mechanisms corresponding to cognitive states.

[0003] The present invention designs learning activities that trigger high- and low-order cognitive states, trains classification models to realize automatic detection of cognitive states, and combines explainable artificial intelligence technology to perform attribution analysis to verify the effectiveness of this experimental method. First, according to Bloom's taxonomy of teaching objectives and ICAP (Interactive–Constructive–Active–Passive) cognitive engagement theory, high- and low-order learning activities are designed and implemented, and EEG data of learners participating in these two types of activities are collected and annotated. Then, based on the CNN-LSTM model, the cognitive states of learners participating in high- and low-order learning activities are classified, and the improved explainable model LIME-BA is studied to perform attribution analysis on high- and low-order cognitive states, and verified by deleting the important and non-important EEG features obtained by the explanation model to verify the credibility of the explanation.

[0004] The results show that the CNN-LSTM model can be applied to cognitive state detection with an accuracy of 99%, which is better than using CNN and LSTM alone. Attribution analysis found that the channels that contribute more to cognitive states in high-order learning activities are mainly distributed in the frontal and temporal regions, while the channels that contribute more to cognitive states in low-order learning activities are mainly distributed in the occipital and parietal regions. The features learned by the classification model are consistent with the findings of cognitive activities in previous brain science research, which verifies the effectiveness of the method in this experiment. Summary of the invention

[0005] In view of this, the present invention provides an automatic detection and attribution analysis method of learners' cognitive state based on EEG.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] The automatic detection and attribution analysis method of the learner's cognitive state based on EEG includes the following steps:

[0008] Step S1: Experimental design

[0009] Design high- and low-level learning activities, recruit subjects, collect their EEG signals when participating in learning activities, obtain EEG data, and construct LADataset;

[0010] Step S2: Cognitive state detection

[0011] Based on the constructed learning activity data, CNN, LSTM, and CNN-LSTM models are used to detect learners’ high- and low-order cognitive states;

[0012] Step S3: Conduct attribution analysis

[0013] For the collected EEG data, each channel is regarded as a feature to be explained. Then, the linear regression model is used to fit the local boundary of the explanation model to obtain the contribution value of each channel of the sample to be explained. Finally, the prediction samples with higher confidence are selected and the average value of their explanation results is calculated.

[0014] Preferably, in step S1, the learning activities include copying learning materials, watching learning videos and drawing concept maps.

[0015] Preferably, in step 1, the low-level learning activities include copying materials and watching videos; the high-level learning activities include drawing concept maps;

[0016] Preferably, in step 1, the cognitive state represented by the EEG signal generated when the learner participates in high-order learning activities is marked as high-order, and the cognitive state represented by the EEG signal when the learner participates in low-order learning activities is marked as low-order.

[0017] Preferably, in step 1, 22 college students, 10 girls and 12 boys, with an average age of 24 years old, who are healthy, have no medical history, and voluntarily participate in the experiment are recruited.

[0018] Preferably, in step 1, the method for collecting EEG signals during learning activities and obtaining EEG data is:

[0019] The experimenter was responsible for assisting the subjects in wearing the experimental equipment, ensuring the stable operation of the EEG equipment, and recording the EEG signals of the 22 subjects as they completed copying materials, watching videos, and drawing concept maps;

[0020] The collected EEG data were preprocessed and downsampled to 128 Hz, and then band-pass filtering was used to retain the 0.5-45 Hz signal. The ICA algorithm was used to remove artifacts caused by eye and muscle movements to obtain EEG data.

[0021] Preferably, in step 1, the method for constructing LADataset is:

[0022] The EEG data were preprocessed and sliced ​​using non-overlapping time windows of one second. Each second of EEG data was a sample, and the size of each sample was 32×128. The high cognitive state group obtained 22,704 samples, and the low cognitive state group obtained 27,192 samples, which together constitute the dataset LADataset.

[0023] Preferably, in step 2, the CNN-LSTM model is mainly composed of an input layer, 3 convolution modules, 1 l stm module, a fully connected module, and a softmax output layer; the CNN block is composed of a maximum pooling layer and Dropout regularization.

[0024] Preferably, in step 3, first, for the collected EEG data, each channel is regarded as a feature to be interpreted, using x∈{0,1} d To represent the binary vector of its interpretable features, where d is 23, corresponding to the channels of the frontal lobe, temporal lobe, occipital lobe, and parietal lobe to be interpreted; each channel is regarded as a feature to be explained, and its original representation is recorded as x∈R 23 ×23×128 ; 1 means that Gaussian noise is added to the corresponding channel for disturbance, and 0 means that Gaussian noise is not added to the corresponding channel; then, the local boundary of the explanation model is fitted using the linear regression model to obtain the contribution value of each channel of the sample to be explained; finally, the prediction sample with higher confidence is selected and the average value of its explanation result is calculated.

[0025] Preferably, in step 3, the calculation method is as follows:

[0026]

[0027] Where n represents the total number of samples with high interpretation confidence, W k It represents the contribution value of the i-th channel in the n-th test sample; the CCD value range is -1 to 1; positive and negative values ​​indicate the direction of the channel's contribution to the model's prediction results; a positive value indicates that the prediction result is consistent with the detection model, and a negative value indicates that it is opposite to the prediction result. The larger the absolute value of CCD, the greater the impact, while a value close to zero or zero indicates a smaller impact.

[0028] Compared with the prior art, the present invention has achieved the following technical effects:

[0029] The accuracy rate of the method of the present invention reaches 99%, and the effectiveness of the method of the present invention is verified. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a process diagram of the automatic detection and attribution analysis method of the EEG learner's cognitive state according to the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The present invention discloses an automatic detection and attribution analysis method of a learner's cognitive state based on EEG, comprising the following steps:

[0033] Step S1: Experimental design

[0034] Design high- and low-level learning activities, recruit subjects, collect their EEG signals when participating in learning activities, obtain EEG data, and construct LADataset;

[0035] Learning activities include copying learning materials, watching learning videos, and drawing concept maps; low-order learning activities include copying materials and watching videos; high-order learning activities include drawing concept maps; the cognitive state represented by the EEG signals generated by learners participating in high-order learning activities is marked as high-order, and the cognitive state represented by the EEG signals when participating in low-order learning activities is marked as low-order;

[0036] We recruited 22 college students, 10 females and 12 males, with an average age of 24 years, who were healthy, had no medical history, and were willing to participate in the experiment;

[0037] The experimenter was responsible for assisting the subjects in wearing the experimental equipment and ensuring the stable operation of the EEG equipment. The EEG signals of the 22 subjects were recorded when they completed copying materials, watching videos, and drawing concept maps. The collected EEG data were preprocessed and downsampled to 128 Hz, and then band-pass filtered to retain the 0.5-45 Hz signal. The I CA algorithm was used to remove artifacts caused by eye and muscle movements to obtain EEG data.

[0038] The EEG data were preprocessed and sliced ​​using non-overlapping time windows of one second. Each second of EEG data was a sample, and the size of each sample was 32×128. The high cognitive state group obtained 22,704 samples, and the low cognitive state group obtained 27,192 samples, which together constituted the data set LADataset;

[0039] Step S2: Cognitive state detection

[0040] Based on the constructed learning activity data, CNN, LSTM, and CNN-LSTM models are used to detect learners’ high- and low-order cognitive states;

[0041] The CNN-LSTM model mainly consists of an input layer, three convolutional modules, an LSTM module, a fully connected module, and a softmax output layer; the CNN block consists of a maximum pooling layer and Dropout regularization;

[0042] Step S3: Conduct attribution analysis

[0043] First, for the collected EEG data, each channel is regarded as a feature to be explained, using x∈{0,1} d To represent the binary vector of its interpretable features, where d is 23, corresponding to the channels of the frontal lobe, temporal lobe, occipital lobe, and parietal lobe to be interpreted; each channel is regarded as a feature to be explained, and its original representation is recorded as x∈R 23×23×128 ; 1 means that Gaussian noise is added to the corresponding channel for perturbation, and 0 means that Gaussian noise is not added to the corresponding channel; then, the local boundary of the explanation model is fitted using the linear regression model to obtain the contribution value of each channel of the sample to be explained; finally, the prediction sample with higher confidence is selected and the average value of its explanation result is calculated;

[0044] The calculation method is as follows:

[0045]

[0046] Where n represents the total number of samples with high interpretation confidence, W k It represents the contribution value of the i-th channel in the n-th test sample; the CCD value range is -1 to 1; positive and negative values ​​indicate the direction of the channel's contribution to the model's prediction results; a positive value indicates that the prediction result is consistent with the detection model, and a negative value indicates that it is opposite to the prediction result. The larger the absolute value of CCD, the greater the impact, while a value close to zero or zero indicates a smaller impact.

[0047] In order to verify the credibility of LIME-BA's explanation of the recognition model, the channel contributions were sorted, and the channel features at the head and tail of the arrangement, that is, important features and unimportant features, were removed respectively. The EEG signals after removal were re-input into the model for recognition to observe whether they had any impact on the accuracy.

[0048] A positive channel contribution indicates that the channel feature helps to identify the cognitive state in the learning activity, and a larger channel contribution indicates a greater impact on the model decision. The influence of brain regions on model decision-making is further determined by the channel distribution position.

[0049] The results show that the CNN-LSTM model is applied to cognitive state detection with an accuracy of 99%.

[0050] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. Automatic detection and attribution analysis method of learner's cognitive state based on EEG, characterized by: The following steps are involved: Step S1: Experimental design Design high- and low-level learning activities, recruit subjects, collect their EEG signals when participating in learning activities, obtain EEG data, and construct LADataset; Step S2: Cognitive state detection Based on the constructed learning activity data, CNN, LSTM, and CNN-LSTM models are used to detect learners’ high- and low-order cognitive states; Step S3: Conduct attribution analysis For the collected EEG data, each channel is regarded as a feature to be explained. Then, the linear regression model is used to fit the local boundary of the explanation model to obtain the contribution value of each channel of the sample to be explained. Finally, the prediction samples with higher confidence are selected and the average value of their explanation results is calculated.

2. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 1, characterized in that: In step S1, the learning activities include copying learning materials, watching learning videos and drawing concept maps.

3. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 2, characterized in that: In step 1, low-level learning activities include copying materials and watching videos; high-level learning activities include drawing concept maps.

4. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 3, characterized in that: In step 1, the cognitive state represented by the EEG signal generated when the learner participates in high-order learning activities is marked as high-order, and the cognitive state represented by the EEG signal when the learner participates in low-order learning activities is marked as low-order.

5. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 1, characterized in that: In step 1, 22 college students (10 girls and 12 boys) with an average age of 24 years old, who were healthy, had no medical history, and voluntarily participated in the experiment were recruited.

6. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 1, characterized in that: In step 1, the method of collecting EEG signals during learning activities and obtaining EEG data is as follows: The experimenter was responsible for assisting the subjects in wearing the experimental equipment, ensuring the stable operation of the EEG equipment, and recording the EEG signals of the 22 subjects as they completed copying materials, watching videos, and drawing concept maps; The collected EEG data were preprocessed and downsampled to 128 Hz, and then band-pass filtering was used to retain the 0.5-45 Hz signal. The ICA algorithm was used to remove artifacts caused by eye and muscle movements to obtain EEG data.

7. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 1, characterized in that: In step 1, the method for constructing LADataset is: The EEG data were preprocessed and sliced ​​using non-overlapping time windows of one second. Each second of EEG data was a sample, and the size of each sample was 32×128. The high cognitive state group obtained 22,704 samples, and the low cognitive state group obtained 27,192 samples, which together constitute the dataset LADataset.

8. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 1, characterized in that: In step 2, the CNN-LSTM model is mainly composed of an input layer, three convolution modules, one LSTM module, a fully connected module, and a softmax output layer; the CNN block is composed of a maximum pooling layer and Dropout regularization.

9. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 1, characterized in that: In step 3, first, for the collected EEG data, each channel is regarded as a feature to be interpreted, using x∈{0,1} d To represent the binary vector of its interpretable features, where d is 23, corresponding to the channels of the frontal lobe, temporal lobe, occipital lobe, and parietal lobe to be interpreted; each channel is regarded as a feature to be explained, and its original representation is recorded as x∈R 23×23×128 ; 1 means that Gaussian noise is added to the corresponding channel for disturbance, and 0 means that Gaussian noise is not added to the corresponding channel; then, the local boundary of the explanation model is fitted using the linear regression model to obtain the contribution value of each channel of the sample to be explained; finally, the prediction sample with higher confidence is selected and the average value of its explanation result is calculated.

10. The method for automatic detection and attribution analysis of learner cognitive state based on EEG according to claim 1, characterized in that: In step 3, the calculation method is as follows: Where n represents the total number of samples with high interpretation confidence, W k It represents the contribution value of the i-th channel in the n-th test sample; the CCD value range is -1 to 1; positive and negative values ​​indicate the direction of the channel's contribution to the model's prediction results; a positive value indicates that the prediction result is consistent with the detection model, and a negative value indicates that it is opposite to the prediction result. The larger the absolute value of CCD, the greater the impact, while a value close to zero or zero indicates a smaller impact.

Images