Learner cognitive state recognition method and device, electronic equipment and storage medium

Abstract

The invention discloses a learner cognitive state recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining electroencephalogram data of a learner in a plurality of learning behavior processes; dividing data of each electrode channel in the preprocessed electroencephalogram data into frequency band data of a plurality of different frequency bands, segmenting the plurality of frequency band data corresponding to each electrode channel according to a preset time window to obtain a plurality of time sequence frequency band data, and extracting a plurality of signal features from all the time sequence frequency band data; constructing four-dimensional time sequence feature data based on the extracted signal features; performing fusion feature representation on the four-dimensional time sequence feature data to obtain a fusion feature vector; and inputting the fusion feature vector into a pre-trained cognitive state recognition model to obtain a cognitive state recognition result of the learner. According to the learner cognitive state recognition method and device, the electronic equipment and the storage medium, the cognitive state of the learner can be accurately recognized.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of educational information technology, and in particular, to a method, device, electronic device, and computer-readable storage medium for identifying a learner's cognitive state. Background Art

[0002] With the continuous evolution of the education model, the cultivation of higher-order thinking ability, as the main goal of education and teaching, is the core of learner ability training. Higher-order thinking ability is crucial for learners' future academic and career development. Based on this, it is necessary to identify the cognitive state of learners. Accurately identifying the cognitive state of learners can help educators adjust teaching content or teaching methods in a timely manner, and can also help educators optimize teaching strategies and develop personalized learning paths to better meet the individual needs of learners. In the prior art, educators usually assess the cognitive state of learners. This method of identifying the cognitive state of learners may fail to accurately identify the cognitive state of learners due to many subjective factors. Summary of the invention

[0003] The embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for identifying a learner's cognitive state, which can achieve the purpose of accurately identifying a learner's cognitive state.

[0004] The present application provides a method for identifying a learner's cognitive state, including:

[0005] Acquiring electroencephalogram (EEG) data of a learner during multiple learning behaviors, wherein the EEG data includes data of multiple electrode channels;

[0006] Preprocessing the electroencephalogram data, dividing the data of each electrode channel in the preprocessed electroencephalogram data into frequency band data of multiple different frequency bands, segmenting the multiple frequency band data corresponding to each electrode channel by a preset time window to obtain time series frequency band data corresponding to each electrode channel, and extracting multiple signal features from all the time series frequency band data;

[0007] Mapping each signal feature extracted from each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then performing concatenation to obtain three-dimensional data, and concatenating all three-dimensional data in a time series to obtain four-dimensional time series feature data;

[0008] Performing fusion feature representation on the four-dimensional time series feature data to obtain a fusion feature vector;

[0009] The fused feature vector is input into a pre-trained cognitive state recognition model to obtain a cognitive state recognition result of the learner.

[0010] Optionally, the extracting multiple signal features from all the time series frequency band data includes:

[0011] Differential entropy features, power spectral density mean features, power spectral density maximum features and energy features are extracted from all the time series frequency band data.

[0012] Optionally, the performing fusion feature representation on the four-dimensional time series feature data includes:

[0013] The four-dimensional time series feature data is convolved using a convolutional network based on depthwise separable convolution.

[0014] Optionally, the electroencephalogram data includes data of 64 electrode channels, and the two-dimensional distribution is data of a 10×11 two-dimensional structure.

[0015] Optionally, the multiple learning behaviors include: watching learning videos, marking article keywords, and writing short articles.

[0016] Optionally, the pre-trained cognitive state recognition model is a pre-trained BiLSTM model.

[0017] Optionally, the preprocessing of the electroencephalogram data includes:

[0018] Performing data segmentation on the electroencephalogram data;

[0019] The segmented EEG data were subjected to electrode localization, re-reference, down-sampling to 128 Hz, and artifact removal operations in sequence.

[0020] The present application also provides a learner cognitive state recognition device, including:

[0021] A data acquisition module, used to acquire EEG data of the learner during multiple learning behaviors, wherein the EEG data includes data of multiple electrode channels;

[0022] A feature organization module is used to preprocess the EEG data, divide the data of each electrode channel in the preprocessed EEG data into frequency band data of multiple different frequency bands, divide the multiple frequency band data corresponding to each electrode channel by a preset time window, obtain the time series frequency band data corresponding to each electrode channel, and extract multiple signal features from all the time series frequency band data; and map each signal feature extracted based on each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then connect them in series to obtain three-dimensional data, connect all three-dimensional data in series in time series to obtain four-dimensional time series feature data; a feature fusion module is used to fuse the time series feature data to obtain a fused feature vector;

[0023] The recognition module is used to input the fused feature vector into a pre-trained cognitive state recognition model to obtain a recognition result of the learner's cognitive state.

[0024] On the one hand, an embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory stores executable program code; the processor coupled to the memory calls the executable program code stored in the memory to execute the learner cognitive state identification method provided in the above embodiment.

[0025] On one hand, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for identifying the cognitive state of a learner provided in the above embodiment is implemented.

[0026] It can be seen from the above embodiments of the present application that by obtaining the EEG data of the learner in multiple learning behavior processes, the EEG data includes data of multiple electrode channels; preprocessing the EEG data, and dividing the data of each electrode channel in the preprocessed EEG data into frequency band data of multiple different frequency bands, dividing the multiple frequency band data corresponding to each electrode channel by a preset time window, obtaining the time series frequency band data corresponding to each electrode channel, and extracting multiple signal features from all the time series frequency band data; mapping each signal feature extracted based on each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then concatenating them to obtain three-dimensional data, concatenating all three-dimensional data in time series to obtain four-dimensional time series feature data; performing fusion feature representation on the four-dimensional time series feature data to obtain a fusion feature vector; inputting the fusion feature vector into a pre-trained cognitive state recognition model to obtain the cognitive state recognition result of the learner. The embodiment of the present application can identify the learner's cognitive state based on the EEG data of the learner in multiple learning behavior processes, avoiding the deviation caused by subjective evaluation, and achieving the purpose of accurately identifying the learner's cognitive state. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1 A flowchart of a method for identifying a learner's cognitive state according to an embodiment of the present application;

[0029] Figure 2A schematic diagram of the structure of a learner cognitive state recognition device provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1 , Figure 1 This is a flowchart of the implementation of the learner cognitive state recognition method provided in one embodiment of the present application. The method described in this embodiment can be applied to electronic devices, which can be smart terminals such as personal computers and servers. Figure 1 As shown, the method specifically includes:

[0033] Step S11, obtaining electroencephalogram data of the learner during multiple learning behaviors, wherein the electroencephalogram data includes data of multiple electrode channels.

[0034] In this embodiment, the learner is a user who is learning. The learner may be a junior or senior student, or an adult learner.

[0035] In this embodiment, the electroencephalogram data is EEG data. Specifically, the electroencephalogram data includes data of multiple electrode channels, for example, data of 64 electrode channels.

[0036] In this embodiment, the multiple learning behaviors may be continuous or discontinuous learning behaviors. The learning objects of the learning behaviors may be the same subject or different subjects. The acquired EEG data may be one segment of EEG data or multiple segments of EEG data.

[0037] Specifically, the plurality of learning behaviors include different types of learning behaviors. For example, the learning behaviors include answering questions and participating in social practices.

[0038] Furthermore, in an optional embodiment of the present invention, the multiple learning behaviors include: watching learning videos, marking article keywords, and writing short articles.

[0039] That is, during the specific implementation, the EEG data of learners are obtained during the three learning behaviors of watching learning videos, marking article keywords, and writing short articles.

[0040] When learners watch learning videos, they achieve the most superficial understanding of knowledge, which is a passive learning behavior; when learners mark article keywords, they achieve a superficial understanding of knowledge, which is an active learning behavior; when learners write short articles, they achieve a deep understanding of knowledge, which is a constructive learning behavior.

[0041] Therefore, by obtaining the EEG data of learners during the three learning behaviors of watching learning videos, marking article keywords, and writing short articles, we can obtain the EEG data of students participating in learning behaviors of different levels and dimensions, which is conducive to enriching the characteristics of the data, enhancing the distinguishability of the model, and improving the accuracy of prediction.

[0042] Furthermore, in an optional embodiment of the present invention, the step of obtaining EEG data of the learner in multiple learning activities includes:

[0043] The EEG data of the learners during multiple learning activities collected by the LiveAmp 64 electrode cap were obtained from the recording software.

[0044] In this embodiment, the recording software is software that can store EEG data. For example, the recording software may be BrainVision Recorder software.

[0045] In this embodiment, the LiveAmp 64 electrode cap has 64 channels, which can provide high-resolution EEG signal recording, making it easier to acquire EEG data and analyze the learner's cognitive state.

[0046] In specific implementation, the EEG data collected by the LiveAmp 64 electrode cap is stored in the recording software, and then the EEG data is directly obtained from the recording software.

[0047] Step S12, preprocessing the EEG data, and dividing the data of each electrode channel in the preprocessed EEG data into frequency band data of multiple different frequency bands, dividing the multiple frequency band data corresponding to each electrode channel according to a preset time window, obtaining the time series frequency band data corresponding to each electrode channel, and extracting multiple signal features from all the time series frequency band data.

[0048] Specifically, preprocessing the EEG data includes performing operations such as baseline correction, segmentation, and data interpolation on the EEG data. In specific implementation, the data of each electrode channel included in the EEG data is preprocessed separately.

[0049] Further, in an optional embodiment of the present invention, the preprocessing of the electroencephalogram data includes:

[0050] Performing data segmentation on the electroencephalogram data;

[0051] The segmented EEG data were subjected to electrode localization, re-reference, down-sampling to 128 Hz, and artifact removal operations in sequence.

[0052] Furthermore, the preprocessing of the EEG data may also include segmentation, electrode positioning, re-referencing, downsampling to 128 Hz, and other operations other than removing artifacts, such as filtering, data normalization, and the like.

[0053] In this embodiment, segmenting the EEG data means marking the acquired EEG data differently.

[0054] Specifically, the EEG data is segmented, including: marking the data of each electrode channel in the EEG data in a first time period from the beginning as data of resting activity, and marking the data of each electrode channel in the EEG data in a second time period before the end as data of learning behavior.

[0055] The values ​​of the first time and the second time are preset thresholds. For example, the data of each electrode channel in the EEG data within 1 minute from the beginning is marked as resting activity data, and the data of each electrode channel in the EEG data within 8 minutes before the end is marked as learning behavior data.

[0056] In this embodiment, the segmented EEG data is sequentially subjected to electrode positioning, re-referencing, downsampling to 128 Hz, and artifact removal operations, and specifically, the data marked as learning behavior is sequentially subjected to electrode positioning, re-referencing, downsampling to 128 Hz, and artifact removal operations. That is, the data marked as learning behavior is subsequently processed to improve data processing efficiency.

[0057] Specifically, the segmented EEG data is sequentially subjected to electrode positioning, re-referencing, and downsampling to 128 Hz, including: loading electrode position information matching the collected EEG data for electrode positioning, inserting a reference electrode for re-referencing, applying a bandpass filter (such as a 0.5-45 Hz bandpass filter) to filter out information in the remaining frequency bands, and downsampling to 128 Hz, and removing artifacts through independent component analysis (ICA).

[0058] In this embodiment, segmentation can be performed to analyze different time periods of EEG data, which is beneficial to improving analysis efficiency; electrode positioning can ensure the accurate position of each electrode, which is helpful to the accuracy of subsequent analysis; re-referencing can ensure the accuracy of data acquisition and improve signal quality; downsampling to 128Hz can reduce the total amount of data, reduce computing load and improve real-time processing capabilities while maintaining important EEG signals; removing artifacts can improve signal quality, which is beneficial to the accuracy of subsequent recognition.

[0059] In this embodiment, the multiple different frequency bands include a Delta frequency band, a Theta frequency band, an Alpha frequency band, a Beta frequency band, and a Gamma frequency band.

[0060] In this embodiment, the preprocessed EEG data is divided into frequency band data of Delta frequency band, frequency band data of Theta frequency band, frequency band data of Alpha frequency band, frequency band data of Beta frequency band and frequency band data of Gamma frequency band. That is, for the data of one electrode channel in the EEG data, frequency band data of 5 different frequency bands can be obtained.

[0061] In this embodiment, the data of one electrode channel can correspond to frequency band data of 5 different frequency bands, and the data of one electrode channel corresponds to 5*N (N is the number obtained by dividing the preset time window) time series frequency band data, then the data of 64 electrode channels contains 64*5 frequency band data of different frequency bands, and the data of 64 electrode channels corresponds to 64*5*N time series frequency band data.

[0062] Further, in an optional embodiment of the present invention, the extracting multiple signal features from all the time series frequency band data includes:

[0063] Differential entropy features, power spectral density mean features, power spectral density maximum features and energy features are extracted from all the time series frequency band data.

[0064] Specifically, the value of the preset time window is a preset value. For example, the data of signals of different frequency bands can be segmented with a time window of T seconds (T is a preset time, such as T is 0.5 or 1). Four features are extracted for each segment (i.e., time series frequency band data), namely, differential entropy feature, power spectral density mean feature, power spectral density maximum feature, and energy feature.

[0065] Specifically, the extraction of differential entropy features, power spectral density mean features, power spectral density maximum features and energy features includes signal segmentation, window processing, Fourier transform, calculation of differential entropy values, calculation of power spectral density mean, calculation of power spectral density maximum, and calculation of energy eigenvalues ​​(which can be calculated by integrating the square of the signal).

[0066] In this embodiment, differential entropy can measure the randomness of the signal, the power spectrum density mean feature can measure how the signal power is distributed with frequency, the power spectrum density maximum feature can indicate the most prominent oscillatory activity in the signal, and energy can reflect the activity intensity of the brain area. Therefore, extracting differential entropy features, power spectrum density mean features, power spectrum density maximum features, and energy features for each time series frequency band data can provide a more comprehensive and accurate description of the EEG data, and can also complement features and fully capture the complexity of brain activity, thereby improving the robustness of the model and the accuracy of model prediction.

[0067] Step S13, mapping each signal feature extracted from each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then connecting them in series to obtain three-dimensional data, and connecting all three-dimensional data in series in time series to obtain four-dimensional time series feature data.

[0068] In this embodiment, the obtained time series characteristic data can be expressed as X n , where X n It is expressed as N×H×W×C, where N represents the number of segmented segments after segmentation with a preset time window, H represents the height of the two-dimensional mapping distribution, W represents the width of the electrode two-dimensional mapping distribution, and C represents the number of features extracted in different frequency bands. n Represents N data consisting of H×W×C.

[0069] Furthermore, in an optional embodiment of the present invention, the electroencephalogram data includes data of 64 electrode channels, and the two-dimensional distribution is data of a 10×11 two-dimensional structure.

[0070] That is, each signal feature extracted based on each frequency band data corresponding to each electrode channel is mapped into two-dimensional data with a two-dimensional distribution of 10×11.

[0071] In this embodiment, the EEG data includes data from 64 electrode channels, which means that when the EEG data is collected, the EEG signals are recorded from 64 different scalp locations. Each electrode channel corresponds to an electrode, thereby detecting electrical activity from the brain.

[0072] For example, after the EEG data is preprocessed and downsampled to 128Hz, the data of each electrode channel in the preprocessed EEG data is divided into frequency bands to obtain Delta (0.5-4Hz) frequency band data, Theta (4-8Hz), Alpha (8-13Hz) frequency band data, Beta (13-30Hz) frequency band data and Gamma (30-45Hz) frequency band data; if the time window Ts is 0.5s, the EEG data of the divided frequency bands is segmented, and the data dimension after segmentation is 64×64 (because if the time window Ts is 0.5 seconds and the sampling rate is is 128Hz, then there are 128×0.5=64128×0.5=64 sampling points in 0.5 seconds, and since the EEG data contains data of 64 electrode channels, the data in each time window will form a vector of 64 electrodes); the differential entropy features, power spectrum density mean features, power spectrum density maximum features, and energy features are extracted from the time segment data of the above five frequency bands (that is, time series frequency band data), and the data dimension is 64×20; the multiple signal features extracted from each time series frequency band data are mapped to a two-dimensional structure of 10×11, and the data dimension is 10×11×20. Finally, the data segments within every 3s are selected for fusion to obtain a four-dimensional data structure with a dimension size of 6×10×11×20, and this data is used as the input data X of the model. n .

[0073] In this embodiment, dimensionality reduction is performed after extracting multiple signal features, which can retain feature richness while reducing redundancy and improving prediction efficiency and accuracy. In addition, experiments have shown that a two-dimensional structure with a two-dimensional distribution of 10×11 can provide a better balance between computational efficiency and required information, better reflect the actual layout of the electrodes, and therefore have a higher prediction accuracy.

[0074] Step S14, performing fusion feature representation on the four-dimensional time series feature data to obtain a fusion feature vector.

[0075] In an optional embodiment, the four-dimensional time series feature data can be fused and represented by a Transformer network to obtain a fused feature vector.

[0076] In another optional embodiment, the four-dimensional time series feature data can be fused and represented by a recurrent neural network to obtain a fused feature vector.

[0077] In another optional embodiment, the four-dimensional time series feature data can also be fused with features through an autoencoder to obtain a fused feature vector.

[0078] In this embodiment, the obtained fused feature vector fuses the time domain (i.e., time series), spatial domain (i.e., electrode spatial features) and frequency domain (i.e., multi-band) features of the EEG data, which is beneficial to improving the richness and accuracy of data representation, thereby improving recognition efficiency.

[0079] Further, in an optional embodiment of the present invention, the fusion feature representation of the four-dimensional time series feature data includes:

[0080] The four-dimensional time series feature data is convolved using a convolutional network based on depthwise separable convolution.

[0081] In this embodiment, the convolutional network based on depthwise separable convolution refers to a convolutional network used to perform depthwise separable operations, and the convolution result obtained by the convolution operation is the fusion feature.

[0082] Specifically, convolution operations are performed through a convolutional network based on depthwise separable convolution, including channel separation through channel-by-channel convolution; the output results of the channel-by-channel convolution are connected through the Concatenate layer, and the connected data is processed by point-by-point convolution. The data after the point-by-point convolution is then flattened through the Flatten layer to obtain a one-dimensional vector and output, and the flattened one-dimensional vector is learned through the Dense layer. Finally, it is reshaped through the Reshape layer.

[0083] For example, for the data X obtained by the above operation n (X n is a four-dimensional data structure, X n It is represented as N×H×W×C, where N=6, H=10, W=11, C=20). Each three-dimensional data block (H×W×C) in the four-dimensional data structure is used as input data to extract features for fusion representation. First, the input data X in (10×11×20) is convolved channel by channel to separate the channels and obtain the data X of each channel. in1 , X in2 …X in20, the data size is (10×11). Then the data of each channel (10×11) is processed by the first Conv convolution layer (64 convolution kernels of size 3×3) of channel-by-channel convolution, and the output data size is (10×11×64). Then, these data are further processed by the second Conv convolution layer (128 convolution kernels of size 3×3), and the output data size is (10×11×128). Then, in the Concatenate layer, the output results of the channel-by-channel convolution of 20 channels are connected, and the output data size is (10×11×2560). Then, these data are processed by point-by-point convolution (64 convolution kernels of size 1×1), and the output data size is (10×11×64). Then, the Flatten layer flattens these data to obtain a one-dimensional vector and outputs it, and the data size is (7040). Then, the Dense layer learns the flattened one-dimensional vector and outputs 512 feature units. Finally, the Reshape layer reshapes the 512 feature units into a tensor of shape (1×512), which is the fused feature vector.

[0084] In this embodiment, when performing a convolution operation through a convolutional network based on deep separable convolution, the features of each input channel can be independently extracted through channel-by-channel convolution, so the features of different channels can be captured, and these channel features can be further combined through point-by-point convolution. In the point-by-point convolution stage, the previously extracted features can be better reused and combined. In addition, deep separable convolution significantly reduces the number of model parameters by separately processing spatial features and cross-channel features, improves computational efficiency, and enhances the real-time performance of operations.

[0085] Step S15, inputting the fused feature vector into a pre-trained cognitive state recognition model to obtain a cognitive state recognition result of the learner.

[0086] In this embodiment, the pre-trained cognitive state recognition model may be a pre-trained RNN model.

[0087] Furthermore, in an optional embodiment of the present invention, the pre-trained cognitive recognition model is a pre-trained BiLSTM model.

[0088] In this embodiment, the BiLSTM model is a bidirectional long short-term memory network. The BiLSTM model can simultaneously consider the information before and after each moment in the sequence data, thereby effectively capturing the state information before and after the current moment. Therefore, the BiLSTM model can better understand the long-term dependencies in the fused features, thereby having better performance when processing the fused features.

[0089] Specifically, supervised learning can be performed on BiLSTM data through training data (i.e., EEG data used for model training, and preprocessing, feature extraction, feature fusion and other operations are performed on the EEG data), and the loss function can be a Categorical_crossentropy loss function.

[0090] In this embodiment, the cognitive state recognition result of the learner refers to which level of the three levels of cognitive states the learner's cognitive state is. Specifically, the three levels of cognitive states include: a first level cognitive state, a second level cognitive state, and a third level cognitive state. Specifically, the first level cognitive state can represent a low cognitive state, the second level cognitive state can represent a medium cognitive state, and the third level cognitive state can represent a high cognitive state.

[0091] In this embodiment, by obtaining the EEG data of the learner during multiple learning behaviors, the EEG data includes data of multiple electrode channels; preprocessing the EEG data, and dividing the data of each electrode channel in the preprocessed EEG data into frequency band data of multiple different frequency bands, dividing the multiple frequency band data corresponding to each electrode channel by a preset time window, obtaining the time series frequency band data corresponding to each electrode channel, and extracting multiple signal features from all the time series frequency band data; mapping each signal feature extracted based on each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then concatenating them to obtain three-dimensional data, concatenating all three-dimensional data in time series to obtain four-dimensional time series feature data; performing fusion feature representation on the four-dimensional time series feature data to obtain a fusion feature vector; inputting the fusion feature vector into a pre-trained cognitive state recognition model to obtain the cognitive state recognition result of the learner. The embodiment of the present application can identify the learner's cognitive state based on the EEG data of the learner during multiple learning behaviors, avoiding the deviation caused by subjective evaluation, and achieving the purpose of accurately identifying the learner's cognitive state.

[0092] See also Figure 2 , a schematic diagram of the structure of a learner cognitive state recognition device provided by an embodiment of the present application. For the convenience of explanation, only the parts related to the embodiment of the present application are shown. The device can be set in an electronic device, which can be a personal computer, a server or other intelligent terminal. The learner cognitive state recognition device includes:

[0093] The data acquisition module 201 is used to acquire the EEG data of the learner during multiple learning behaviors, and the EEG data includes data of multiple electrode channels.

[0094] In this embodiment, the learner is a user who is learning. The learner may be a junior or senior student, or an adult learner.

[0095] In this embodiment, the electroencephalogram data is EEG data. Specifically, the electroencephalogram data includes data of multiple electrode channels, for example, data of 64 electrode channels.

[0096] In this embodiment, the multiple learning behaviors may be continuous or discontinuous learning behaviors. The learning objects of the learning behaviors may be the same subject or different subjects. The acquired EEG data may be one segment of EEG data or multiple segments of EEG data.

[0097] Specifically, the plurality of learning behaviors include different types of learning behaviors. For example, the learning behaviors include answering questions and participating in social practices.

[0098] Furthermore, in an optional embodiment of the present invention, the multiple learning behaviors include: watching learning videos, marking article keywords, and writing short articles.

[0099] That is, during the specific implementation, the EEG data of learners are obtained during the three learning behaviors of watching learning videos, marking article keywords, and writing short articles.

[0100] When learners watch learning videos, they achieve the most superficial understanding of knowledge, which is a passive learning behavior; when learners mark article keywords, they achieve a superficial understanding of knowledge, which is an active learning behavior; when learners write short articles, they achieve a deep understanding of knowledge, which is a constructive learning behavior.

[0101] Therefore, by obtaining the EEG data of learners during the three learning behaviors of watching learning videos, marking article keywords, and writing short articles, we can obtain the EEG data of students participating in learning behaviors of different levels and dimensions, which is conducive to enriching the characteristics of the data, enhancing the distinguishability of the model, and improving the accuracy of prediction.

[0102] Furthermore, in an optional embodiment of the present invention, the data acquisition module 201 is specifically used for:

[0103] The EEG data of the learners during multiple learning activities collected by the LiveAmp 64 electrode cap were obtained from the recording software.

[0104] In this embodiment, the recording software is software that can store EEG data. For example, the recording software may be BrainVision Recorder software.

[0105] In this embodiment, the LiveAmp 64 electrode cap has 64 channels, which can provide high-resolution EEG signal recording, making it easier to acquire EEG data and analyze the learner's cognitive state.

[0106] In specific implementation, the EEG data collected by the LiveAmp 64 electrode cap is stored in the recording software, and then the EEG data is directly obtained from the recording software.

[0107] The feature organization module 202 is used to preprocess the EEG data, and divide the data of each electrode channel in the preprocessed EEG data into frequency band data of multiple different frequency bands, divide the multiple frequency band data corresponding to each electrode channel according to a preset time window, obtain the time series frequency band data corresponding to each electrode channel, and extract multiple signal features from all the time series frequency band data.

[0108] Specifically, preprocessing the EEG data includes performing operations such as baseline correction, segmentation, and data interpolation on the EEG data. In specific implementation, the data of each electrode channel included in the EEG data is preprocessed separately.

[0109] Further, in an optional embodiment of the present invention, the preprocessing of the electroencephalogram data includes:

[0110] Performing data segmentation on the electroencephalogram data;

[0111] The segmented EEG data were subjected to electrode localization, re-reference, down-sampling to 128 Hz, and artifact removal operations in sequence.

[0112] Furthermore, the preprocessing of the EEG data may also include segmentation, electrode positioning, re-referencing, downsampling to 128 Hz, and other operations other than removing artifacts, such as filtering, data normalization, and the like.

[0113] In this embodiment, segmenting the EEG data means marking the acquired EEG data differently.

[0114] Specifically, the EEG data is segmented, including: marking the data of each electrode channel in the EEG data in a first time period from the beginning as data of resting activity, and marking the data of each electrode channel in the EEG data in a second time period before the end as data of learning behavior.

[0115] The values ​​of the first time and the second time are preset thresholds. For example, the data of each electrode channel in the EEG data within 1 minute from the beginning is marked as resting activity data, and the data of each electrode channel in the EEG data within 8 minutes before the end is marked as learning behavior data.

[0116] In this embodiment, the segmented EEG data is sequentially subjected to electrode positioning, re-referencing, downsampling to 128 Hz, and artifact removal operations, and specifically, the data marked as learning behavior is sequentially subjected to electrode positioning, re-referencing, downsampling to 128 Hz, and artifact removal operations. That is, the data marked as learning behavior is subsequently processed to improve data processing efficiency.

[0117] Specifically, the segmented EEG data is sequentially subjected to electrode positioning, re-referencing, and downsampling to 128 Hz, including: loading electrode position information matching the collected EEG data for electrode positioning, inserting a reference electrode for re-referencing, applying a bandpass filter (such as a 0.5-45 Hz bandpass filter) to filter out information in the remaining frequency bands, and downsampling to 128 Hz, and removing artifacts through independent component analysis (ICA).

[0118] In this embodiment, segmentation can be performed to analyze different time periods of EEG data, which is beneficial to improving analysis efficiency; electrode positioning can ensure the accurate position of each electrode, which is helpful to the accuracy of subsequent analysis; re-referencing can ensure the accuracy of data acquisition and improve signal quality; downsampling to 128Hz can reduce the total amount of data, reduce computing load and improve real-time processing capabilities while maintaining important EEG signals; removing artifacts can improve signal quality, which is beneficial to the accuracy of subsequent recognition.

[0119] In this embodiment, the multiple different frequency bands include a Delta frequency band, a Theta frequency band, an Alpha frequency band, a Beta frequency band, and a Gamma frequency band.

[0120] In this embodiment, the preprocessed EEG data is divided into frequency band data of the Delta frequency band, frequency band data of the Theta frequency band, frequency band data of the Alpha frequency band, frequency band data of the Beta frequency band, and frequency band data of the Gamma frequency band. That is, for the data of one electrode channel in the EEG data, frequency band data of 5 different frequency bands can be obtained. In this embodiment, the data of one electrode channel can correspond to the frequency band data of 5 different frequency bands, and the data of one electrode channel corresponds to 5*N (N is the number obtained by dividing by a preset time window) time series frequency band data, then the data of 64 electrode channels contains 64*5 frequency band data of different frequency bands, and the data of 64 electrode channels corresponds to 64*5*N time series frequency band data.

[0121] Further, in an optional embodiment of the present invention, the extracting multiple signal features from all the time series frequency band data includes:

[0122] Differential entropy features, power spectral density mean features, power spectral density maximum features and energy features are extracted from all the time series frequency band data.

[0123] Specifically, the value of the preset time window is a preset value. For example, the data of signals of different frequency bands can be segmented with a time window of T seconds (T is a preset time, such as T is 0.5 or 1). Four features are extracted for each segment (i.e., time series frequency band data), namely, differential entropy feature, power spectral density mean feature, power spectral density maximum feature, and energy feature.

[0124] Specifically, the extraction of differential entropy features, power spectral density mean features, power spectral density maximum features and energy features includes signal segmentation, window processing, Fourier transform, calculation of differential entropy values, calculation of power spectral density mean, calculation of power spectral density maximum, and calculation of energy eigenvalues ​​(which can be calculated by integrating the square of the signal).

[0125] In this embodiment, differential entropy can measure the randomness of the signal, the power spectrum density mean feature can measure how the signal power is distributed with frequency, the power spectrum density maximum feature can indicate the most prominent oscillatory activity in the signal, and energy can reflect the activity intensity of the brain area. Therefore, extracting differential entropy features, power spectrum density mean features, power spectrum density maximum features, and energy features for each time series frequency band data can provide a more comprehensive and accurate description of the EEG data, and can also complement features and fully capture the complexity of brain activity, thereby improving the robustness of the model and the accuracy of model prediction.

[0126] The feature organization module 202 is also used to map each signal feature extracted from each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then connect them in series to obtain three-dimensional data, and connect all three-dimensional data in series in time series to obtain four-dimensional time series feature data.

[0127] In this embodiment, the obtained time series characteristic data can be expressed as X n , where X n It is expressed as N×H×W×C, where N represents the number of segmented segments after segmentation with a preset time window, H represents the height of the two-dimensional mapping distribution, W represents the width of the electrode two-dimensional mapping distribution, and C represents the number of features extracted in different frequency bands. n Represents N data consisting of H×W×C.

[0128] Furthermore, in an optional embodiment of the present invention, the electroencephalogram data includes data of 64 electrode channels, and the two-dimensional distribution is data of a 10×11 two-dimensional structure.

[0129] That is, each signal feature extracted based on each frequency band data corresponding to each electrode channel is mapped into two-dimensional data with a two-dimensional distribution of 10×11.

[0130] In this embodiment, the EEG data includes data from 64 electrode channels, which means that when the EEG data is collected, the EEG signals are recorded from 64 different scalp locations. Each electrode channel corresponds to an electrode, thereby detecting electrical activity from the brain.

[0131] For example, after the EEG data is preprocessed and downsampled to 128Hz, the data of each electrode channel in the preprocessed EEG data is divided into frequency bands to obtain Delta (0.5-4Hz) frequency band data, Theta (4-8Hz), Alpha (8-13Hz) frequency band data, Beta (13-30Hz) frequency band data and Gamma (30-45Hz) frequency band data; if the time window Ts is 0.5s, the EEG data of the divided frequency bands is segmented, and the data dimension after segmentation is 64×64 (because if the time window Ts is 0.5 seconds and the sampling rate is is 128Hz, then there are 128×0.5=64128×0.5=64 sampling points in 0.5 seconds, and since the EEG data contains data of 64 electrode channels, the data in each time window will form a vector of 64 electrodes); the differential entropy features, power spectrum density mean features, power spectrum density maximum features, and energy features are extracted from the time segment data of the above five frequency bands (that is, time series frequency band data), and the data dimension is 64×20; the multiple signal features extracted from each time series frequency band data are mapped to a two-dimensional structure of 10×11, and the data dimension is 10×11×20. Finally, the data segments within every 3s are selected for fusion to obtain a four-dimensional data structure with a dimension size of 6×10×11×20, and this data is used as the input data X of the model. n .

[0132] In this embodiment, dimensionality reduction is performed after extracting multiple signal features, which can retain feature richness while reducing redundancy and improving prediction efficiency and accuracy. In addition, experiments have shown that a two-dimensional structure with a two-dimensional distribution of 10×11 can provide a better balance between computational efficiency and required information, better reflect the actual layout of the electrodes, and therefore have a higher prediction accuracy.

[0133] The feature fusion module 203 is used to fuse the four-dimensional time series feature data to obtain a fused feature vector.

[0134] In an optional embodiment, the four-dimensional time series feature data can be fused and represented by a Transformer network to obtain a fused feature vector.

[0135] In another optional embodiment, the four-dimensional time series feature data can be fused and represented by a recurrent neural network to obtain a fused feature vector.

[0136] In another optional embodiment, the four-dimensional time series feature data can also be fused with features through an autoencoder to obtain a fused feature vector.

[0137] In this embodiment, the obtained fused feature vector fuses the time domain (i.e., time series), spatial domain (i.e., electrode spatial features) and frequency domain (i.e., multi-band) features of the EEG data, which is beneficial to improving the richness and accuracy of data representation, thereby improving recognition efficiency.

[0138] Furthermore, in an optional embodiment of the present invention, the feature fusion module 204 is specifically used for:

[0139] The four-dimensional time series feature data is convolved using a convolutional network based on depthwise separable convolution.

[0140] In this embodiment, the convolutional network based on depthwise separable convolution refers to a convolutional network used to perform depthwise separable operations, and the convolution result obtained by the convolution operation is the fusion feature.

[0141] Specifically, convolution operations are performed through a convolutional network based on depthwise separable convolution, including channel separation through channel-by-channel convolution; the output results of the channel-by-channel convolution are connected through the Concatenate layer, and the connected data is processed by point-by-point convolution. The data after the point-by-point convolution is then flattened through the Flatten layer to obtain a one-dimensional vector and output, and the flattened one-dimensional vector is learned through the Dense layer. Finally, it is reshaped through the Reshape layer.

[0142] For example, for the data X obtained by the above operation n (X n is a four-dimensional data structure, X n It is represented as N×H×W×C, where N=6, H=10, W=11, C=20). Each three-dimensional data block (H×W×C) in the four-dimensional data structure is used as input data to extract features for fusion representation. First, the input data X in (10×11×20) is convolved channel by channel to separate the channels and obtain the data X of each channel. in1 , X in2 …X in20, the data size is (10×11). Then the data of each channel (10×11) is processed by the first Conv convolution layer (64 convolution kernels of size 3×3) of channel-by-channel convolution, and the output data size is (10×11×64). Then, these data are further processed by the second Conv convolution layer (128 convolution kernels of size 3×3), and the output data size is (10×11×128). Then, in the Concatenate layer, the output results of the channel-by-channel convolution of 20 channels are connected, and the output data size is (10×11×2560). Then, these data are processed by point-by-point convolution (64 convolution kernels of size 1×1), and the output data size is (10×11×64). Then, the Flatten layer flattens these data to obtain a one-dimensional vector and outputs it, and the data size is (7040). Then, the Dense layer learns the flattened one-dimensional vector and outputs 512 feature units. Finally, the Reshape layer reshapes the 512 feature units into a tensor of shape (1×512), which is the fused feature vector.

[0143] In this embodiment, when performing a convolution operation through a convolutional network based on deep separable convolution, the features of each input channel can be independently extracted through channel-by-channel convolution, so the features of different channels can be captured, and these channel features can be further combined through point-by-point convolution. In the point-by-point convolution stage, the previously extracted features can be better reused and combined. In addition, deep separable convolution significantly reduces the number of model parameters by separately processing spatial features and cross-channel features, improves computational efficiency, and enhances the real-time performance of operations.

[0144] The recognition module 204 is used to input the fused feature vector into a pre-trained cognitive state recognition model to obtain a recognition result of the learner's cognitive state.

[0145] In this embodiment, the pre-trained cognitive state recognition model may be a pre-trained RNN model.

[0146] Furthermore, in an optional embodiment of the present invention, the pre-trained cognitive recognition model is a pre-trained BiLSTM model.

[0147] In this embodiment, the BiLSTM model is a bidirectional long short-term memory network. The BiLSTM model can simultaneously consider the information before and after each moment in the sequence data, thereby effectively capturing the state information before and after the current moment. Therefore, the BiLSTM model can better understand the long-term dependencies in the fused features, thereby having better performance when processing the fused features.

[0148] Specifically, supervised learning can be performed on BiLSTM data through training data (i.e., EEG data used for model training, and preprocessing, feature extraction, feature fusion and other operations are performed on the EEG data), and the loss function can be a Categorical_crossentropy loss function.

[0149] In this embodiment, the cognitive state recognition result of the learner refers to which level of the three levels of cognitive states the learner's cognitive state is. Specifically, the three levels of cognitive states include: a first level cognitive state, a second level cognitive state, and a third level cognitive state. Specifically, the first level cognitive state can represent a low cognitive state, the second level cognitive state can represent a medium cognitive state, and the third level cognitive state can represent a high cognitive state.

[0150] In this embodiment, by obtaining the EEG data of the learner during multiple learning behaviors, the EEG data includes data of multiple electrode channels; preprocessing the EEG data, and dividing the data of each electrode channel in the preprocessed EEG data into frequency band data of multiple different frequency bands, dividing the multiple frequency band data corresponding to each electrode channel by a preset time window, obtaining the time series frequency band data corresponding to each electrode channel, and extracting multiple signal features from all the time series frequency band data; mapping each signal feature extracted based on each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then concatenating them to obtain three-dimensional data, concatenating all three-dimensional data in time series to obtain four-dimensional time series feature data; performing fusion feature representation on the four-dimensional time series feature data to obtain a fusion feature vector; inputting the fusion feature vector into a pre-trained cognitive state recognition model to obtain the cognitive state recognition result of the learner. The embodiment of the present application can identify the learner's cognitive state based on the EEG data of the learner during multiple learning behaviors, avoiding the deviation caused by subjective evaluation, and achieving the purpose of accurately identifying the learner's cognitive state.

[0151] See also Figure 3 , a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application.

[0152] Exemplarily, the electronic device can be any of a mobile or portable computer system device. Specifically, the electronic device can be a mobile phone or smart phone (e.g., a phone based on iPhone TM, an Android TM), a portable game device (e.g., Nintendo DS TM, PlayStation Portable TM, Gameboy Advance TM, iPhone TM), a laptop computer, a PDA, a portable Internet device, and a data storage device, other handheld devices, and electronic devices such as watches, headphones, pendants, earphones, etc.

[0153] The electronic device can also be any one of a plurality of electronic devices, including but not limited to cellular phones, smart phones, other wireless communication devices, personal digital assistants, audio players, other media players, music recorders, video recorders, cameras, other media recorders, radios, medical equipment, vehicle transportation instruments, calculators, programmable remote controls, pagers, laptop computers, desktop computers, printers, netbook computers, personal digital assistants (PDAs), and portable multimedia players (PMPs).

[0154] In some cases, the electronic device can perform multiple functions (e.g., play music, display video, store pictures, and receive and send phone calls). If desired, the electronic device can be a portable device such as a cellular phone, a media player, other handheld device, or other compact portable device.

[0155] like Figure 3 As shown, the electronic device 10 may include a control circuit, which may include a storage and processing circuit 30. The storage and processing circuit 30 may include a memory, such as a hard disk drive memory, a non-volatile memory (such as a flash memory or other electronically programmable memory with limited deletion for forming a solid-state drive, etc.), a volatile memory (such as a static or dynamic random access memory, etc.), etc., which is not limited in the embodiment of the present application. The processing circuit in the storage and processing circuit 30 can be used to control the operation of the electronic device 10. The processing circuit can be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application-specific integrated circuits, display driver integrated circuits, etc.

[0156] The storage and processing circuit 30 can be used to run software in the electronic device 10, such as an Internet browsing application, a Voice over Internet Protocol (VOIP) phone call application, an email application, a media playback application, an operating system function, etc. These software can be used to perform some control operations, such as image acquisition based on a camera, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display functions based on status indicators such as status indicators of light-emitting diodes, touch event detection based on a touch sensor, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in the electronic device 10, etc., which are not limited by the embodiments of the present application.

[0157] Furthermore, the memory stores an executable program code, and a processor coupled to the memory calls the executable program code stored in the memory to execute the above Figure 1 The learner's cognitive state identification method is described in the illustrated embodiment.

[0158] The executable program code includes the above Figure 2 The various modules in the learner cognitive state recognition device described in the illustrated embodiment are, for example, a data acquisition module, a feature organization module and a recognition module.

[0159] The electronic device 10 may further include an input-output circuit 42. The input-output circuit 42 may be used to enable the electronic device 10 to input and output data, that is, to allow the electronic device 10 to receive data from an external device and also to allow the electronic device 10 to output data from the electronic device 10 to an external device. The input-output circuit 42 may further include a sensor 32. The sensor 32 may include an ambient light sensor, a proximity sensor based on light and capacitance, a touch sensor (e.g., based on a light touch sensor and / or a capacitive touch sensor, wherein the touch sensor may be part of a touch display screen or may be used independently as a touch sensor structure), an acceleration sensor, and other sensors.

[0160] The input-output circuit 42 may also include one or more displays, such as the display 14. The display 14 may include a liquid crystal display, an organic light-emitting diode display, an electronic ink display, a plasma display, a display using other display technologies, or a combination of several of them. The display 14 may include a touch sensor array (i.e., the display 14 may be a touch-controlled display screen). The touch sensor may be a capacitive touch sensor formed by an array of transparent touch sensor electrodes (e.g., indium tin oxide (ITO) electrodes), or may be a touch sensor formed using other touch technologies, such as sonic touch, pressure-sensitive touch, resistive touch, optical touch, etc., which is not limited in the embodiments of the present application.

[0161] The electronic device 10 may also include an audio component 36. The audio component 36 may be used to provide audio input and output functions for the electronic device 10. The audio component 36 in the electronic device 10 may include a speaker, a microphone, a buzzer, a tone generator, and other components for generating and detecting sound.

[0162] The communication circuit 38 can be used to provide the electronic device 10 with the ability to communicate with external devices. The communication circuit 38 may include analog and digital input / output interface circuits, and wireless communication circuits based on radio frequency signals and / or optical signals. The wireless communication circuit in the communication circuit 38 may include a radio frequency transceiver circuit, a power amplifier circuit, a low noise amplifier, a switch, a filter, and an antenna. For example, the wireless communication circuit in the communication circuit 38 may include a circuit for supporting near field communication (NFC) by transmitting and receiving near field coupled electromagnetic signals. For example, the communication circuit 38 may include a near field communication antenna and a near field communication transceiver. The communication circuit 38 may also include a cellular phone transceiver and antenna, a wireless local area network transceiver circuit and antenna, etc.

[0163] The electronic device 10 may further include a battery, power management circuits and other input / output units 40. The input / output units 40 may include buttons, joysticks, click wheels, scroll wheels, touch pads, keypads, keyboards, cameras, light emitting diodes and other status indicators.

[0164] A user may input commands through the input-output circuit 42 to control the operation of the electronic device 10 , and may use the output data of the input-output circuit 42 to receive status information and other outputs from the electronic device 10 .

[0165] Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, which may be disposed in the electronic device in the above embodiments. The computer-readable storage medium may be the above Figure 3The memory in the storage and processing circuit 30 in the illustrated embodiment. The computer readable storage medium stores a computer program, which, when executed by the processor, implements the aforementioned Figure 1 The learner cognitive state identification method described in the illustrated embodiment. Further, the computer storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0166] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0167] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0168] The above is a description of the learner cognitive state identification method, device and computer-readable storage medium provided by the present invention. For a person skilled in the art, according to the ideas of the embodiments of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for identifying a learner's cognitive state, characterized in that: The method comprises: Acquiring electroencephalogram (EEG) data of a learner during multiple learning behaviors, wherein the EEG data includes data of multiple electrode channels; Preprocessing the electroencephalogram data, dividing the data of each electrode channel in the preprocessed electroencephalogram data into frequency band data of multiple different frequency bands, segmenting the multiple frequency band data corresponding to each electrode channel by a preset time window to obtain time series frequency band data corresponding to each electrode channel, and extracting multiple signal features from all the time series frequency band data; Mapping each signal feature extracted from each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then performing concatenation to obtain three-dimensional data, and concatenating all three-dimensional data in a time series to obtain four-dimensional time series feature data; Performing fusion feature representation on the four-dimensional time series feature data to obtain a fusion feature vector; The fused feature vector is input into a pre-trained cognitive state recognition model to obtain a cognitive state recognition result of the learner.

2. The method according to claim 1, characterized in that The extracting of multiple signal features from all the time series frequency band data includes: Differential entropy features, power spectral density mean features, power spectral density maximum features and energy features are extracted from all the time series frequency band data.

3. The method according to claim 1, characterized in that The performing fusion feature representation on the four-dimensional time series feature data includes: The four-dimensional time series feature data is convolved using a convolutional network based on depthwise separable convolution.

4. The method according to claim 1, characterized in that The electroencephalogram data includes data of 64 electrode channels, and the two-dimensional distribution is data of a 10×11 two-dimensional structure.

5. The method according to claim 1, characterized in that The multiple learning behaviors include: watching learning videos, marking article keywords, and writing short articles.

6. The method according to claim 1, characterized in that The pre-trained cognitive state recognition model is a pre-trained BiLSTM model.

7. The method according to any one of claims 1 to 6, characterized in that The preprocessing of the electroencephalogram data comprises: Performing data segmentation on the electroencephalogram data; The segmented EEG data were subjected to electrode localization, re-reference, down-sampling to 128 Hz, and artifact removal operations in sequence.

8. A learner cognitive state recognition device, characterized in that: The device comprises: A data acquisition module, used to acquire EEG data of the learner during multiple learning behaviors, wherein the EEG data includes data of multiple electrode channels; A feature organization module is used to preprocess the EEG data, divide the data of each electrode channel in the preprocessed EEG data into frequency band data of multiple different frequency bands, divide the multiple frequency band data corresponding to each electrode channel by a preset time window, obtain the time series frequency band data corresponding to each electrode channel, and extract multiple signal features from all the time series frequency band data; and map each signal feature extracted based on each frequency band data corresponding to each electrode channel into a two-dimensional distribution and then connect them in series to obtain three-dimensional data, connect all the three-dimensional data in series in time series to obtain four-dimensional time series feature data; A feature fusion module, used for performing fusion feature representation on the time series feature data to obtain a fusion feature vector; The recognition module is used to input the fused feature vector into a pre-trained cognitive state recognition model to obtain a recognition result of the learner's cognitive state.

9. An electronic device, characterized in that: The electronic device comprises: Memory and processor; The memory stores executable program code; The processor coupled to the memory calls the executable program code stored in the memory to execute the learner cognitive state identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying a learner's cognitive state as claimed in any one of claims 1 to 7 is implemented.

Images