Metacognition training method based on electroencephalogram data and deep learning model

Abstract

The present application relates to a kind of based on electroencephalogram data and deep learning model's meta-cognition training method, comprising: through electroencephalogram equipment, the brain cognitive information data of participant is collected to carry out data preprocessing;Through fourier transform, the original electroencephalogram data obtained is transferred to the energy value on the preset frequency domain wave band, and the brain cognitive information data is obtained by calculating timing energy value;The brain cognitive data set obtained is divided into label set and feature set to carry out data marking and screening;According to brain cognitive preset training course, on-site teaching is carried out, and the training state and brain cognitive data form of participant are observed and recorded, form the label set corresponding to brain cognitive data set, for the model training of next step;LSTM network model is constructed, and brain cognitive data set is proportionally split into training set and test set to carry out model training;The training model currently obtained is verified and model is corrected, and the state of participant is automatically marked using training model, and report is generated for reference.

Description

Technical Field

[0001] This invention relates to the field of neuroscience, and more particularly to the field of brain cognitive information processing, specifically to a metacognitive training method based on electroencephalogram (EEG) data and deep learning models. Background Technology

[0002] Metacognition is defined as cognition of cognition, that is, self-awareness and self-regulation of cognitive activities. It includes people's awareness of their own cognitive abilities, their awareness of specific tasks, and their awareness of different strategies. Research suggests that "metacognition is essentially a set of explicit and declarative processes that depend on the interviewee's ability to report their psychological experiences." Traditional research methods in this field include metacognitive knowledge interviews and self-report. Their advantages are: low cost and the ability to achieve in-depth understanding through readily adjustable questions; disadvantages include: self-report bias; significant influence from the interviewer's personal ability; and unsuitability for individuals who cannot accurately describe their behavior, such as young children.

[0003] Meanwhile, the education field lacks a technology capable of quantifying and presenting metacognitive processes. Most applications rely on experienced teachers, and while metacognitive-based teaching methods are praised, they are difficult to promote. Current technologies include solutions for online metacognitive assessment, but these rely on software-recorded student behavioral data. However, if the scenario is transferred to a wider offline environment, this technology loses its data source. Furthermore, behavioral data is not intuitive in reflecting a person's inner world and has limited ability to describe "psychological experiences"; moreover, it is often mixed with noisy data that is non-specific and difficult to remove, thus affecting accuracy. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a metacognitive training method based on EEG data and deep learning models that is timely, easy to implement, and produces more accurate training results.

[0005] To achieve the above objectives, the metacognitive training method based on EEG data and deep learning models of the present invention is as follows:

[0006] This metacognitive training method based on EEG data and deep learning models is characterized by the following steps:

[0007] (1) Collect the trainees’ current brain cognitive information data through EEG devices and perform corresponding EEG data preprocessing;

[0008] (2) By using Fourier transform, the currently acquired raw EEG data is transferred to a preset frequency domain band, and the energy value dataset is calculated. Based on the energy value dataset, the brain cognition dataset is obtained.

[0009] (3) The obtained brain cognition dataset is divided into a label set and a feature set for data labeling and filtering, classification is completed, and a new feature set for system prediction is obtained.

[0010] (4) Conduct on-site teaching according to the brain cognition pre-set training course, observe and record the current training status and brain cognition data form of the trainees, form a label set corresponding to the obtained brain cognition dataset, and use it for model training in the next step;

[0011] (5) Construct an LSTM network model and split the brain cognition dataset into training and testing sets according to the proportions for model training;

[0012] (6) Validate the currently acquired training model. If it does not meet the validation criteria, supplement the data, retrain the model, and make corrections until it passes the validation.

[0013] (7) Visualize the optimized brain cognitive information data to obtain labeled information and visualized brain cognitive status reports that reflect the trainees’ metacognitive abilities, so that trainees can review and improve themselves.

[0014] Preferably, step (2) includes the following steps:

[0015] (2.1) The host computer software uses Fourier transform to convert the raw EEG data in the current moving time window into five frequency domain bands, including alpha, beta, gamma, delta and theta waves, and calculates the corresponding energy values. Finally, the energy values ​​are transferred to the newly created energy data matrix in chronological order.

[0016] (2.2) The amplitude of the trainee's movement is determined based on the acceleration data obtained from the accelerometer on the EEG device. Then, based on the δ-wave energy value in the energy data matrix, the mass qlty of the electrode corresponding to each data point at each movement time window is calculated. sec = [q1, q2, q3, …, q elc ], where q elc This indicates the number of electrodes in the EEG device. After the data collection is completed, the EEG data quality dataset qlty is obtained.

[0017] Bad data (including poor contact, excessive motion interference, and excessive eye movement) were removed from the obtained EEG data quality dataset qlty, resulting in a focus dataset attCourse, an emotion information dataset emoCourse, and an active thinking dataset memCourse with a time resolution of 1 second. The datasets collected in subsequent courses were then set as cognInd = {attCourse, emoCourse, memCourse}.

[0018] Preferably, step (3) sets the tag set L in the following manner:

[0019] L = {attL, emoL, memL}

[0020] Among them, attL is the focus label set, emoL is the emotional information label set, and memL is the active thinking label set;

[0021] The feature set is set in the following manner in step (3):

[0022] cognFea = {attFea, emoFea, memFea}

[0023] The feature set here is obtained from the dataset cognInd collected from any course, where cognFea is the initial feature set, attFea is the focus feature set, emoFea is the emotion information feature set, and memFea is the active thinking feature set.

[0024] Principal component analysis was used to map the brain cognitive feature set cognFea from its original d-dimensional features to a lower dimension, specifically:

[0025] Any subset of the brain cognitive feature set cognFea is represented as ,in, n for The first in n A subset To perform the transpose operation, The dimension is n×d, and the data will be data-driven as follows:

[0026] ;

[0027] in, It is centralized ; It is the first k Feature set corresponding to each feature n This represents the sample size.

[0028] The covariance matrix of a feature is calculated as follows:

[0029] ;

[0030] Calculate the correlation coefficient matrix:

[0031] in, , and These represent two positions symmetrical along the diagonal, and the eigenvalues ​​of the correlation coefficient matrix are... Number of principal components q To be set as:

[0032] ;

[0033] in, It is the eigenvalue corresponding to the k-th principal component; It can be any one of the principal components;

[0034] The eigenvectors are calculated as follows:

[0035] ;

[0036] in, d This represents the original number of features, i.e., the number of features contained in cognFea. q This indicates the number of features contained in the new feature set pFea after filtering. It represents any eigenvector.

[0037] Preferably, step (3) further includes:

[0038] After principal component analysis, sensitive features selected from the brain cognitive feature set cognFea are used to form a new feature set pFea = {attPrn, emoPrn, memPrn} for subsequent system prediction. Here, attPrn is the new focus feature set, emoPrn is the new emotional information feature set, and memPrn is the new active thinking feature set.

[0039] Based on the data of multiple people taking the same course stored in the system norm library, the threshold set mkTH = {aH,aL, eH, eL, wH, wL} is calculated, where aH and aL are the high and low thresholds of focus; eH and eL are the high and low thresholds of emotional information; and wH and wL are the high and low thresholds of active thinking.

[0040] Preferably, the brain cognition preset training course described in step (4) specifically includes:

[0041] The training course consists of n lessons plus one verification lesson, denoted as {cl1, cl2, cl3, ..., cl...}. j Let}, 1≤j≤n, where j is the j-th lesson. The n lessons are divided into three types of training courses according to their quantity. The data obtained from the type 1 training course is used to train the model. The type 1 training course further divides each lesson into phase 1 and phase 2 for differentiated training. The data obtained from the type 2 training course is used for the validation of the training model, and metacognitive training is conducted for the trainees in the current course phase. The type 3 training course is used to automatically conduct targeted training based on the trainees' individual circumstances.

[0042] Preferably, step (4) further includes data tagging and segmentation in the following manner:

[0043] Trainers observe and record the participants' state and cognitive data patterns on-site. After each lesson, trainers communicate with participants to review the entire lesson and record the cognitive state during key time periods. Combining on-site observation and post-lesson communication, a label set is generated. The start time of the cognitive dataset is recorded as 0 seconds, and every s seconds thereafter, the temporal cognitive data is divided into a segment epoch = {ep1, ep2, ... ep}. e There are a total of e segments and epochNums sub-segments, where epochNums = [h / s], and h is the class duration and s is the slice duration, represented as {ep e , e∈epochNums};When h is not divisible by s, epochNums only takes the integer part, and the data of the remainder period is discarded;

[0044] The labels confirmed by the label set L are used to form a label set cnMarker = {aMarker, eMarker, wMarker} corresponding to the brain cognitive dataset cognTrain. Here, aMarker is the set of attention labels, eMarker is the set of emotion information labels, and wMarker is the set of active thinking labels. Any element in any of the label sets aMarker, eMarker, and wMarker is denoted by mkrTRNe, {mkrTRNe, e∈epochNums}. Each of them corresponds to cognitive data of duration s. cognTrain and cnMarker will be used for the next step of model training.

[0045] Preferably, step (5) constructs an LSTM network model according to the encoding layer, input gate, forget gate, hidden state, and output layer, wherein the ten-fold cross-validation method is processed as follows:

[0046] (5.1) The brain cognition dataset cognTrain is split into training set and test set according to the proportion. Both are normalized and then input into the LSTM network model together with the label information obtained from the current label set L.

[0047] (5.2) Adjust the hidden layer size, learning rate, optimizer, loss function, and number of samples in each batch. Repeat the adjustment of parameters until the loss coefficient is minimized to complete the model training for the trainee.

[0048] Preferably, step (6) specifically includes:

[0049] Through the verification course, all task types set in the Type 1 training course stage 2 are verified, and the data in the verification course is labeled according to the data labeling method in step (4) to obtain the verification set cognTest. If the verification is qualified, the current model is put into use, and the system automatically labels the classroom data after each course, starting from the Type 2 training course. If the verification is unqualified, the data needs to be extracted from the Type 2 training course according to the proportion, added to the brain cognition dataset cognTrain, and the model is trained again until the training is qualified.

[0050] Set up a confusion matrix CM for the label set with dimensions qL×qL. The index of any predicted value from the model is denoted as i, and the index of any actual class is denoted as j. Then verify the accuracy. Represented as:

[0051] ;

[0052] Where qL is the length (6 or 8) of any item attL, emoL, memL in the label set L, and the accuracy of the model is evaluated by comparing it with a preset threshold.

[0053] Preferably, step (7) includes the following steps:

[0054] (7.1) After each Type II and Type III training course, the dataset cognInd after model training is obtained. j Where j is the j-th lesson;

[0055] (7.2) The system processes the dataset cognInd j After normalization and fragment partitioning, the fragment partitioning dataset classNrmEp is obtained. j ,classNrmEp j= {attEpoch, emoEpoch, memEpoch}, where attEpoch is the dataset for attention segmentation, emoEpoch is the dataset for emotion information segmentation, and memEpoch is the dataset for active thinking segmentation;

[0056] (7.3) According to class NrmEp j The model automatically labels the brain's cognitive state throughout the entire metacognitive training session of lesson j using the obtained data, resulting in the labeled dataset cognMarker. j = {attMkr j emoMkr j memMkr j}, where attMkr j For the focus-labeled dataset, emoMkr j For emotion information labeling datasets, memMkr j To proactively think about labeled datasets, attMkr j emoMkr j memMkr j Any element in any set is represented as mkr e ,{mkr e , e∈epochNums}, automatically generated by the LSTM model every s seconds;

[0057] (7.4) After the marking is completed, the system will automatically group logically identical and consecutive time periods of mkr e The tags are chained together to form an identifier corresponding to the timeline of cognitive changes in the brain, represented as the set cognSign = {aSign, eSign, wSign}, where aSign is the identifier of focus level, eSign is the identifier of emotional information, and wSign is the identifier of active thinking; and a certain mkr is judged according to the following method. e Are they connected to each other's adjacent labels?

[0058] ;

[0059] in, , Refers to the labels that mark a series of events;

[0060] (7.5) Visualize the set cognSign = {aSign, eSign, wSign} and the dataset cognInd = {attCourse, emoCourse, memCourse} in time synchronization to generate a brain cognitive state report.

[0061] This invention employs a metacognitive training method based on EEG data and a deep learning model. Addressing the weaknesses of current metacognitive teaching methods, it quantifies the crucial metacognitive ability dimension of "metacognitive experience" in real time based on brain cognitive information data and neuroscience algorithms. This data is then fed into a deep learning model, which automatically labels key data segments and generates reports, quantifying the "metacognitive regulatory effect." This lowers the barrier to entry for after-class metacognitive teaching tutoring, making it accessible even to subject teachers lacking brain and metacognitive experience. Trainees learn to adjust their cognitive strategies and processes based on guidance from after-class reports. For educational institutions, this facilitates metacognitive teaching for subject teachers and reduces labor costs. Attached Figure Description

[0062] Figure 1 This is a flowchart of the metacognitive training method based on EEG data and deep learning models of the present invention.

[0063] Figure 2 This is an architecture diagram of the metacognitive training method based on EEG data and deep learning models of the present invention. Detailed Implementation

[0064] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0065] Before describing the embodiments of the present invention in detail, it should be noted that, in the following, the terms “comprising,” “including,” or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0066] This technical solution presents a portable system for metacognitive training based on EEG data and deep learning models, applicable in various scenarios including daily life, study, and work. It consists of two parts: 1. Multiple portable EEG acquisition devices and accompanying host computer software, responsible for collecting raw EEG data and performing calculations to obtain cognitive information; 2. A cloud system that, based on each individual's cognitive information, trains deep learning models and provides real-time feedback during metacognitive training tasks.

[0067] Please see Figure 1 and 2 As shown below, this technical solution will be explained in further detail:

[0068] I. Obtaining Brain Cognitive Information Datasets from EEG Data

[0069] 1. One or more participants wear EEG devices, and the EEG data is transmitted via Bluetooth and received by the host computer software. The sampling rate of each channel is 256 Hz, and all EEG signals are referenced at Fpz. Interference from electrooculography (EOG) and electromyography (EMG) is removed, and the data is processed by a Butterworth filter with a cutoff frequency between 0.5 and 40 Hz.

[0070] The length of the moving time window is movgT, and the moving interval is 1 second. The start time of EEG data acquisition is recorded as 0 seconds. After accumulating movgT data, the software outputs the first one-dimensional array of energy values. Then the time window moves forward by 1 second and outputs the second one-dimensional array of energy values. This process continues until the acquisition ends.

[0071] The host computer software uses Fourier transform to convert the raw EEG data within the time window into five frequency domain bands (alpha, beta, gamma, delta, and theta waves) and calculates the energy values. These energy values ​​are then stored in a newly created energy data matrix in chronological order.

[0072] The EEG device has a built-in accelerometer. Based on the accelerometer data, it is determined that the participant's movement amplitude is too large. This is then supplemented by the delta wave energy value in the aforementioned matrix to calculate the mass qlty at each time window. sec After the data collection was completed, the EEG data quality dataset qlty was obtained; qlty sec Represented as: qlty sec = [q1, q2, q3, …, q elc ].

[0073] qlty sec Each element in the equation corresponds to an electrode and the signal quality at the current time point; elc represents the number of electrodes in the EEG device used; q elc The range and meaning of the values ​​are as follows:

[0074] •1, Good signal

[0075] •-1, Some data is affected by eye movements, but is still usable.

[0076] •-2, The electrode is in contact with the skin, but the connection is abnormal.

[0077] •-3, severely affected by eye movement, data unavailable.

[0078] •-4, Data is significantly affected by head movement.

[0079] •-5, the signal is severely interfered with.

[0080] •-6, The corresponding electrode is not in contact with the skin.

[0081] The software extracts energy values ​​from the energy data matrix for calculation, removes bad data based on the values ​​in qlty, and obtains datasets attCourse, emoCourse, and memCourse with a time resolution of 1 second. These correspond to three brain cognitive information data: Attention, Emotion, and Thinking, respectively.

[0082] EEG data collection is conducted in the classroom, in any course. j The corresponding dataset is represented as cognInd j Hereinafter referred to as cognInd, it is represented as:

[0083] cognInd = {attCourse, emoCourse, memCourse}

[0084] Course cl j The duration is h minutes; the length of each of the three items contained in cognInd is len seconds. , Unit: seconds.

[0085] In this example, movgT = 10 seconds; h = 30 minutes.

[0086] In practical applications, examples of removing bad data are as follows:

[0087] Example a: Data from a triaxial accelerometer needs to be obtained from an EEG device to calculate the amplitude of motion. If it exceeds a threshold, it is marked as excessive motion interference.

[0088] Example b: Extract electrooculography (EOG) data from the frontal EEG electrode data, calculate the eye movement amplitude, and mark it as excessive eye movement interference if it exceeds the threshold.

[0089] II. Labeling of Brain Cognitive Information Data

[0090] 1. Tag set

[0091] The label set is denoted as L, and the label sets for focus, emotional information, and active thinking are attL, emoL, and memL, respectively; below, cognL will be used to refer to any one of them; represented as: L = {attL, emoL, memL}

[0092] All terms in attL and memL, and the first six terms in emoL, are: rise, fall, maintain, oscillate, rise above the threshold, and fall below the threshold.

[0093] The last two terms in emoL are: positive to negative and negative to positive.

[0094] 2. Feature set

[0095] Calculate cognInd to obtain the initial feature set cognFea, represented as: cognFea = {attFea, emoFea, memFea}

[0096] The three subsets correspond to focus, emotional information, and active thinking, respectively. The built-in feature names in cognFea are as follows: Time-domain features: mean amplitude, variance, coefficient of variation, mode, maximum value, median, minimum value, peak-to-peak value, number of positive values, number of negative values, percentage of positive values, standard deviation, quartiles, kurtosis, slope, similarity, zero-crossing rate, mean deviation, skewness, kurtosis, etc.; Frequency-domain features: mean frequency, centroid frequency, frequency standard deviation, etc.

[0097] 3. Feature Filtering

[0098] Principal Component Analysis (PCA) is used to map the brain cognitive feature set cognFea from the original d-dimensional features to a lower dimension. This reduces the number of features and improves data processing speed.

[0099] Any subset of cognFea is represented as The input feature set is represented as:

[0100] ;

[0101] The dimension is ;

[0102] Datafication:

[0103] ;

[0104] Calculate the covariance matrix of a certain feature:

[0105] ;

[0106] The dimension is ;

[0107] Correlation coefficient matrix:

[0108] ;

[0109] in ;

[0110] Eigenvalues ​​of the correlation coefficient matrix:

[0111] ;

[0112] Determine the number of principal components q:

[0113] ;

[0114] Calculate the corresponding eigenvectors:

[0115] ;

[0116] Calculate the obtained principal components: ;

[0117] 4. Obtain the feature set for prediction

[0118] After PCA processing, the sensitive features selected from cognFea form a new feature set pFea, which is expressed as:

[0119] pFea = {attPrn, emoPrn, memPrn};

[0120] attPrn = {aF1, aF2, aF3};

[0121] emoPrn = {eF11, eF12, eF13, eF14, eF24};

[0122] memPrn = {wF1, wF2, wF3};

[0123] Among them, aF1, eF11, wF1 represent the amplitude average value; aF2, eF12, wF2 represent the variance; aF3, eF13, wF3 represent the similarity; eF14 represents the zero-crossing rate; eF24 represents the positive value ratio.

[0124] For a time series signal with a time resolution of 1 second and a duration of s (i.e., ep in Section 2 of the third step e ), it is expressed as: cng = {cng1, cng2, cng3…, cng s};

[0125] Mean value:

[0126] ;

[0127] Variance:

[0128] ;

[0129] Similarity: freDist = {fre1, fre2, fre3…, fre x}, x < m, indicating that the similarity is not calculated for each label l m All.

[0130] fre x It is the label (l) of each element in cognInd and its corresponding item (cognL) in set L. m The similarity between characteristic waveforms.

[0131] Zero crossing rate: Represents the number of times the signal cng crosses the zero point.

[0132] Positive value percentage: This represents the ratio of the number of data points with values ​​greater than zero in the signal cng to the total number of data points. Its minimum value is 0, and its maximum value is 1.

[0133] 5. Obtain the threshold set

[0134] The system norm library contains data on multiple people taking the same course, and the threshold set mkTH is calculated.

[0135] mkTH = {aH, aL, eH, eL, wH, wL}

[0136] Among them, aH and aL are the high and low thresholds of focus; eH and eL are the high and low thresholds of emotional information; and wH and wL are the high and low thresholds of active thinking.

[0137] III. Model Training

[0138] 1. Course Description

[0139] In one specific embodiment, the method is designed as 40 lessons, represented as {cl1, cl2, cl3, ..., cl j}, 1≤j≤40. In addition to the 40 lessons, there is one verification lesson. Lessons 1 to 5 are for training the model (i.e., Type 1 training course); lessons 5 to 10 are for verifying and revising the model (i.e., Type 2 training course), while also providing metacognitive training to the participants; lessons 11 to 40 (i.e., Type 3 training course) are for the system to automatically conduct targeted training based on the participants' individual circumstances.

[0140] The first 5 lessons:

[0141] Lesson 1, Stage 1: Study as usual; Stage 2: Add metacognitive awareness to the learning process.

[0142] Lesson 2, Phase 1: Learning-related tasks; Phase 2: Non-learning tasks.

[0143] Lesson 3, Phase 1: Things you're interested in; Phase 2: Things you're not interested in.

[0144] Lesson 4, Phase 1: Tasks you are good at; Phase 2: Tasks you are not good at.

[0145] Lesson 5, Phase 1: Tasks that do not cause stress; Phase 2: Tasks that cause stress.

[0146] Training data:

[0147] The datasets from lessons 1 to 5, i.e., the data from the model training phase, are merged for training. They are then stacked row by row as shown in the following formula:

[0148] ;

[0149] 2. Data tagging and segmentation

[0150] Trainers observe and record the trainees' state and cognitive data patterns on-site. After each class, trainers communicate with trainees to review the entire class and record the cognitive state during key time periods. Combining the above two assessments, a tag set is formed.

[0151] The dataset begins at time 0 seconds. Every s seconds thereafter, the temporal cognitive data is divided into an epoch. {ep1, ep2, ... ep...} e There are a total of epochNums sub-segments, where epochNums = [h / s], which can be represented as {ep e , e∈epochNums}; when h is not divisible by s, epochNums only takes the integer part, and the data of the remainder period is discarded.

[0152] In this example, s = 10 seconds;

[0153] The verified labels form a label set corresponding to the cognTrain brain cognition dataset:

[0154] cnMarker = {aMarker, eMarker, wMarker}

[0155] aMarker, eMarker, and wMarker represent sets of labels for attention, emotional information, and active thinking data, respectively.

[0156] Any element from any of the tag sets aMarker, eMarker, or wMarker, using mkrTRN e This indicates that {mkrTRN e , e∈epochNums}; each of them corresponds to cognitive data of duration s;

[0157] The element mkrTRN in aMarker and wMarker e The forms are the same, which is l. mIt includes multiple forms.

[0158] cognTrain and cnMarker will be used for the next step of model training.

[0159] IV. Constructing an LSTM Network Model

[0160] 1. LSTM network model

[0161] Encoding layer:

[0162] An LSTM network consists of: a sequence input layer, an LSTM layer, a fully connected layer, a softmax layer, and a classification output layer.

[0163] Input Gate:

[0164] ;

[0165] ;

[0166] in, i t Input gate; [h t-1 ,x t The hidden state of the previous time point and the input of the current time point; σ It is an activation function; the LSTM layer uses the sigmoid function by default. Cell t represents a new memory unit; tanh represents the hyperbolic tangent function, with an output range between -1 and +1.

[0167] Forgotten Gate:

[0168] ;

[0169] in, f t Forgotten Gate; W f The weight matrix representing the forget gate; b f The offset of the forget gate;

[0170] The internal state of a memory cell:

[0171] ;

[0172] Where ⊙ represents the Hadamard product. It is the memory unit at the current time t;

[0173] Hidden state:

[0174] Input gate i tControl through the new memory unit Cell t How much information is extracted from new data; the forgetting gate f t The decision was made in the previous memory unit. How much information is retained?

[0175] ;

[0176] It is the memory unit at the current time t. ;

[0177] Output layer:

[0178] ;

[0179] in, o t Output gate; W o The weight matrix representing the output gate; b o The offset of the output gate;

[0180] 2. Conduct training

[0181] The training employed 10-fold cross-validation.

[0182] The cognTrain is split into training and testing sets proportionally; both sets are normalized and then fed into the Long Short-Term Memory (LSTM) network model along with the labeled information.

[0183] Training and parameter tuning. Adjust parameters such as hidden layer size (hidden_size), learning rate (learning_rate), optimizer (optimizer), loss function (criterion), and number of samples in each batch (batch_size).

[0184] Repeatedly adjust the parameters until the loss coefficient is minimized to complete the model training for this participant. If the model is not satisfactory after the validation in step five, you can choose to supplement the data using the method in step five and train the model again according to the above steps.

[0185] V. Validation and Model Refinement

[0186] The verification lessons cover all the task types that appear in Phase 2, from lessons 1 to 5.

[0187] After the validation lesson ends, the data from the validation lesson is labeled using the same method as in step three, and used as the validation set cognTest. If the validation meets the requirements, the model is put into use, and starting from lesson 6, the classroom data will be automatically labeled by the system after the course ends.

[0188] If the model fails the verification, data must be extracted proportionally from lessons 6 to 10, added to cognTrain, and then the model must be trained again until it passes the training test.

[0189] Lesson 6 corresponds to Lesson 1; Task: Metacognitive observation.

[0190] Lesson 7 corresponds to Lesson 2; Task: Can learning be made easier and more enjoyable?

[0191] Lesson 8 corresponds to Lesson 3; Task: Can you make yourself more interested?

[0192] Lesson 9 corresponds to Lesson 4; Task: Find a sense of ease and proficiency in the task.

[0193] Lesson 10, corresponding to Lesson 5; Task: Can you improve your state of tension?

[0194] The validation metric, ACC value, is expressed as:

[0195] ;

[0196] Where TP (True Positive) indicates that the actual test result is consistent with the classification prediction result and is positive; FP (False Positive) indicates that the actual test result is negative and the prediction result is positive; TN (True Negative) indicates that the actual test result is consistent with the classification prediction result and is negative; FN (False Negative) indicates that the actual test result is positive and the prediction result is negative.

[0197] The confusion matrix CM of the label set has dimensions qL×qL. attL and memL contain six terms, emoL contains eight terms, and qL takes values ​​of 6 or 8 respectively. The index of any predicted value is i, and the index of any actual class is j. Based on the above formula, the accuracy can be expressed as:

[0198] ;

[0199] The accuracy of the model is evaluated by comparing the ACC with the preset threshold.

[0200] VI. Brain Cognitive Status Report

[0201] 1. Curriculum and Metacognitive Teaching Process

[0202] Each class lasts h minutes. After the class begins, brain cognitive data is sent to the model, and the system helps trainees conduct metacognitive training through fully automated "real-time guidance information".

[0203] From lessons 6 to 40, after each lesson, we obtain the dataset cognInd. j .

[0204] 2. Automatically tag classroom data:

[0205] After the course is completed, the model labels the data from this lesson and converts the labels into identifiers with a longer coverage period.

[0206] The system uses cognInd j After normalization and fragmentation, the dataset classNrmEp is obtained. j The method for segmenting is the same as in step 3, section 2, with each epoch lasting s seconds.

[0207] classNrmEp j = {attEpoch, emoEpoch, memEpoch}

[0208] For ease of reference below, any one of attEpoch, emoEpoch, and memEpoch will be referred to as cognEpoch.

[0209] According to classNrmEp j The model automatically labels the brain's cognitive state throughout the entire metacognitive training session of lesson j, using the data in the dataset. This results in a labeled dataset, denoted as cognMarker. j .

[0210] cognMarker j = {attMkr j emoMkr j memMkr j}

[0211] attMkr j emoMkr j memMkr j Any element in any set is represented as mkr e ,{mkr e ,e∈epochNums}, is automatically generated by the LSTM model every s seconds.

[0212] 3. Labeling brain cognitive information

[0213] mkr e The corresponding data duration is seconds (s), but the actual duration of changes in brain cognitive states during the course usually exceeds seconds. After labeling, the system automatically groups logically identical and consecutively timed MKR data. eThe tags are concatenated to form an identifier corresponding to the time course of brain cognitive changes, denoted as the set cognSign.

[0214] cognSign = {aSign, eSign, wSign}

[0215] Among them, aSign, eSign, and wSign are all sets, corresponding to the identifier information of concentration, emotional information, and active thinking respectively.

[0216] The content of the identifier information includes:

[0217] Independent changes, for any one of concentration, emotional information, and active thinking: 1. Mark the events of its significant increase and significant decrease with time points; 2. Mark the time periods of its remaining at a high level and remaining at a low level with start and end times;

[0218] Linked changes, 1. Mark the synchronous changes of any two of concentration, emotional information, and active thinking with time periods; 2. Mark the regulatory effect of emotional information on the other two with time periods.

[0219] Determine a certain mkr e The logic of whether it is connected to its adjacent tag:

[0220] ;

[0221] {cnctInd k , k<epochNums}, 1<intvl<epochNums, where intvl = 2;

[0222] In Section 1 of the third step, the tag set L is defined, and the l included in L m has various forms; taking "rise, fall, maintain, oscillate, rise above the threshold, fall below the threshold" as an example;

[0223] After "rise above the threshold" appears, the identifier information (cognSign) of this segment is set to "remain at a high level", and the start timestamp is recorded; other tags are compatible with "remain at a high level", cnctInd k = 1, and the identifier information remains unchanged; "remain at a high level" and "fall below the threshold" are heterogeneous, cnctInd k = 0, and the end timestamp is recorded.

[0224] After "fall below the threshold" appears, the identifier information of this segment is set to "remain at a low level", and the start timestamp is recorded; other tags are compatible with "remain at a low level", cnctInd k = 1, and the identifier information remains unchanged; "remain at a low level" and "rise above the threshold" are heterogeneous, cnctInd k= 0, record the end timestamp.

[0225] The "Maintain High Level" indicator can be mapped as follows: for focus, it can be mapped as "focused state"; for emotional information, it can be mapped as "high emotional valence"; and for active thinking, it can be mapped as "thinking state".

[0226] The "Keep Low" label can be mapped to "non-focused state" for focus; "low emotional valence" for emotional information; and "non-thinking state" for active thinking.

[0227] Interconnected changes:

[0228] (1) Any two of the three items aSign, eSign, and wSign in cognSign are marked as "maintaining high position" in the same time period, and this time period can be marked as "covariant - maintain high position"; if they are marked as "maintaining low position" in the same time period, they can be marked as "covariant - maintain high position".

[0229] (2) When emoMkr j When the value "oscillates", then use the emoMkr value. j The start timestamp is 5 seconds above the start time. Within the time frame of `tmSeg`, the sets `aSign` and `wSign` are searched. If `aSign` has a "maintain high" or "maintain low" flag, it is marked as "moderating focus," and the end time of this flag in `aSign` is used as the end timestamp. `wSign` is searched in the same way to determine whether it is marked as "moderating active thinking." `tmSeg` is set to the end time of "oscillation" plus 20 seconds. If multiple `emoMkr` values ​​are present... j The "oscillations" are linked together, with the end time of the last "oscillation" label as the reference point;

[0230] In the post-class report, cognSign and brain cognitive information cognInd are visualized synchronously over time; when reviewing classroom performance, changes in focus, emotions, and proactive thinking are presented to teachers and trainees in a simple and clear form; these changes actually reflect the trainees' metacognitive observation and adjustment process.

[0231] In practical applications, this technical solution has the following technical effects:

[0232] (1) Save manpower

[0233] This technical solution uses a deep learning model to automatically mark the segments that are related to "metacognitive perception and regulation" and require special attention, based on the trainee's cognitive state.

[0234] Previously, the report interpretation session required trainers with a deep understanding of metacognitive teaching to assess participants' "metacognitive experience and regulation" based on brain cognitive information curves. Automatic tagging allows teachers with extensive teaching experience but lacking knowledge of brain science and metacognition to provide after-class tutoring. For participants aged 12 and above who have completed 10 hours of metacognitive training, automatic tagging also enables them to accurately assess their performance in tasks, laying the foundation for a "home-based training, home-school collaboration" model.

[0235] This technical solution can address the shortage of qualified trainers and also reduce labor costs to some extent.

[0236] (2) It can be used for group training in real-world scenarios.

[0237] This technical solution can collect EEG data from multiple people simultaneously, making it suitable for multi-person communication application scenarios; it is also suitable for embedding into real-life, learning, and work scenarios for testing and training.

[0238] (3) The records are objective and detailed.

[0239] Indicators derived from EEG data can accurately and comprehensively record changes in participants' cognitive activity throughout the task; the records accurately correspond to the participants' different stages (different sub-tasks) in the classroom in terms of time. They are unaffected by participants' self-report bias or memory bias during the post-lesson review phase.

[0240] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0241] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution device.

[0242] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0243] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0244] In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0245] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

[0246] This invention employs a metacognitive training method based on EEG data and a deep learning model. Addressing the weaknesses of current metacognitive teaching methods, it quantifies the crucial metacognitive ability dimension of "metacognitive experience" in real time based on brain cognitive information data and neuroscience algorithms. This data is then fed into a deep learning model, which automatically labels key data segments and generates reports, quantifying the "metacognitive regulatory effect." This lowers the barrier to entry for after-class metacognitive teaching tutoring, making it accessible even to subject teachers lacking brain and metacognitive experience. Trainees learn to adjust their cognitive strategies and processes based on guidance from after-class reports. For educational institutions, this facilitates metacognitive teaching for subject teachers and reduces labor costs.

[0247] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A metacognitive training method based on EEG data and a deep learning model, characterized in that, The method includes the following steps: (1) Collect the trainees’ current brain cognitive information data through EEG devices and perform corresponding EEG data preprocessing; (2) By using Fourier transform, the currently acquired raw EEG data is transferred to a preset frequency domain band, and the energy value dataset is calculated. Based on the energy value dataset, the brain cognition dataset is obtained. (3) The obtained brain cognition dataset is divided into a label set and a feature set for data labeling and filtering, classification is completed, and sensitive features are selected from the brain cognition feature set to form a new feature set for system prediction. (4) Conduct on-site teaching according to the brain cognition pre-set training course, observe and record the current training status and brain cognition data form of the trainees, and form a label set corresponding to the obtained brain cognition dataset; (5) Construct an LSTM network model and split the brain cognition dataset into training and testing sets according to the proportions for model training; (6) Validate the currently acquired training model. If it does not meet the validation criteria, supplement the data, retrain the model, and make corrections until it passes the validation. (7) Visualize the optimized brain cognitive information data to obtain labeled information and visualized brain cognitive status reports that reflect the trainees' metacognitive abilities, so that trainees can review and improve themselves. Step (2) includes the following steps: (2.1) The host computer software uses Fourier transform to convert the raw EEG data in the current moving time window into five frequency domain bands, including alpha, beta, gamma, delta and theta waves, and calculates the corresponding energy values. Finally, the energy values ​​are transferred to the newly created energy data matrix in chronological order. (2.2) The amplitude of the trainee's movement is determined based on the acceleration data obtained from the accelerometer on the EEG device. Then, based on the δ-wave energy value in the energy data matrix, the mass qlty of the electrode corresponding to each data point at each movement time window is calculated. sec = [q1, q2, q3, …, q elc ], where q elc This indicates the number of electrodes in the EEG device. After the data collection is completed, the EEG data quality dataset qlty is obtained. Bad data were removed from the obtained EEG data quality dataset qlty, resulting in a focus dataset attCourse, an emotion information dataset emoCourse, and an active thinking dataset memCourse with a time resolution of 1 second. The datasets collected in subsequent courses were then set as cognInd = {attCourse, emoCourse, memCourse}.

2. The metacognitive training method based on EEG data and deep learning models according to claim 1, characterized in that, Step (3) involves setting the tag set L as follows: L = {attL, emoL, memL} Among them, attL is the focus label set, emoL is the emotional information label set, and memL is the active thinking label set; The feature set is set in the following manner in step (3): cognFea = {attFea, emoFea, memFea} The feature set here is obtained from the dataset cognInd collected from any course, where cognFea is the initial feature set, attFea is the focus feature set, emoFea is the emotion information feature set, and memFea is the active thinking feature set. Principal component analysis was used to map the brain cognitive feature set cognFea from its original d-dimensional features to a lower dimension, specifically: Any subset of the brain cognitive feature set cognFea is represented as , where n is The nth subset in To perform the transpose operation, The dimension is n×d, and the data will be data-driven as follows: in, It is centralized , The feature set corresponding to the k-th feature is n, where n is the sample size. The covariance matrix of a feature is calculated as follows: Calculate the correlation coefficient matrix: in, , and These represent two positions symmetrical along the diagonal, and the eigenvalues ​​of the correlation coefficient matrix are... The number of principal components q is set as follows: in, It is the eigenvalue corresponding to the k-th principal component. It can be any one of the principal components; The eigenvectors are calculated as follows: Where d represents the original number of features, i.e., the number of features contained in cognFea, and q represents the number of features contained in the new feature set pFea after filtering. It represents any eigenvector.

3. The metacognitive training method based on EEG data and deep learning models according to claim 2, characterized in that, Step (3) further includes: After principal component analysis, sensitive features selected from the brain cognitive feature set cognFea are used to form a new feature set pFea = {attPrn, emoPrn, memPrn} for subsequent system prediction. Here, attPrn is the new focus feature set, emoPrn is the new emotional information feature set, and memPrn is the new active thinking feature set. Based on the data of multiple people taking the same course stored in the system norm library, the threshold set mkTH = {aH, aL, eH, eL, wH, wL} is calculated, where aH and aL are the high and low thresholds of focus; eH and eL are the high and low thresholds of emotional information; and wH and wL are the high and low thresholds of active thinking.

4. The metacognitive training method based on EEG data and deep learning models according to claim 3, characterized in that, The brain cognition preset training course mentioned in step (4) is specifically as follows: The training course consists of n lessons plus one verification lesson, denoted as {cl1, cl2, cl3, ..., cl...}. j Let}, 1≤j≤n, where j is the j-th lesson. The n lessons are divided into three types of training courses according to their quantity. The data obtained from the type 1 training courses are used to train the model. The type 1 training courses also divide each lesson into phase 1 and phase 2 for differentiated training. The data obtained from the type 2 training courses are used for the validation of the training model, and metacognitive training is carried out for the trainees in the current course phase. The type 3 training courses are used to automatically carry out targeted training based on the trainees' own situation.

5. The metacognitive training method based on EEG data and deep learning models according to claim 4, characterized in that, Step (4) further includes data tagging and segmentation in the following manner: Trainers observe and record the participants' state and cognitive data patterns on-site. After each lesson, trainers communicate with participants to review the entire lesson and record the cognitive state during key time periods. Combining on-site observation and post-lesson communication, a label set is generated. The start time of the cognitive dataset is recorded as 0 seconds, and every s seconds thereafter, the temporal cognitive data is divided into a segment epoch = {ep1, ep2, ... ep}. e There are a total of e segments and epochNums sub-segments, where epochNums = [h / s], and h is the class duration and s is the slice duration, represented as {ep e , e∈epochNums};When h is not divisible by s, epochNums only takes the integer part, and the data of the remainder period is discarded; The labels confirmed by the label set L are used to form a label set cnMarker = {aMarker, eMarker, wMarker} corresponding to the brain cognitive dataset cognTrain. Here, aMarker is the set of attention labels, eMarker is the set of emotion information labels, and wMarker is the set of active thinking labels. Any element in any of the label sets aMarker, eMarker, and wMarker is denoted by mkrTRNe, {mkrTRNe, e∈epochNums}. Each of them corresponds to cognitive data of duration s. cognTrain and cnMarker will be used for the next step of model training.

6. The metacognitive training method based on EEG data and deep learning models according to claim 5, characterized in that, Step (5) constructs an LSTM network model according to the encoding layer, input gate, forget gate, hidden state, and output layer. The ten-fold cross-validation method is processed as follows: (5.1) The brain cognition dataset cognTrain is split into training set and test set according to the proportion. Both are normalized and then input into the LSTM network model together with the label information obtained from the current label set L. (5.2) Adjust the hidden layer size, learning rate, optimizer, loss function, and number of samples in each batch. Repeat the adjustment of parameters until the loss coefficient is minimized to complete the model training for the trainee.

7. The metacognitive training method based on EEG data and deep learning models according to claim 6, characterized in that, The specific steps (6) are as follows: Through the verification course, all task types set in the Type 1 training course are verified, and the data in the verification course are labeled according to the data labeling method in step (4) to obtain the verification set cognTest. If the verification is qualified, the current model is put into use, and the classroom data is automatically labeled by the system after each course, starting from the Type 2 training course. If the verification is unqualified, the data needs to be extracted from the Type 2 training course according to the proportion, added to the brain cognition dataset cognTrain, and the model is trained again until the training is qualified. Set up a confusion matrix CM for the label set with dimensions qL×qL. The index of any predicted value from the model is denoted as i, and the index of any actual class is denoted as j. Then verify the accuracy. Represented as: Where qL is the length of any item attL, emoL, memL in the label set L, and the accuracy of the model is evaluated by comparing it with a preset threshold.

8. The metacognitive training method based on EEG data and deep learning models according to claim 7, characterized in that, Step (7) includes the following steps: (7.1) After each Type II and Type III training course, the dataset cognInd after model training is obtained. j Where j is the j-th lesson; (7.2) The system processes the dataset cognInd j After normalization and fragment partitioning, the fragment partitioning dataset classNrmEp is obtained. j ,classNrmEp j = {attEpoch, emoEpoch, memEpoch}, where attEpoch is the dataset for attention segmentation, emoEpoch is the dataset for emotion information segmentation, and memEpoch is the dataset for active thinking segmentation; (7.3) According to class NrmEp j The model automatically labels the brain's cognitive state throughout the entire metacognitive training session of lesson j using the obtained data, resulting in the labeled dataset cognMarker. j = {attMkr j emoMkr j memMkr j }, where attMkr j For the focus-labeled dataset, emoMkr j For emotion information labeling datasets, memMkr j To proactively think about labeled datasets, attMkr j emoMkr j memMkr j Any element in any set is represented as mkr e ,{mkr e , e∈epochNums}, automatically generated by the LSTM model every s seconds; (7.4) After the marking is completed, the system will automatically group logically identical and consecutive time periods of mkr e The tags are chained together to form an identifier corresponding to the timeline of cognitive changes in the brain, represented as the set cognSign = {aSign, eSign, wSign}, where aSign is the identifier of focus level, eSign is the identifier of emotional information, and wSign is the identifier of active thinking; and a certain mkr is judged according to the following method. e Are they connected to each other's adjacent labels? in,{ , k < epochNums}, 1 < intvl < epochNums, Refers to the labels that mark a series of events; (7.5) Visualize the set cognSign = {aSign, eSign, wSign} and the dataset cognInd = {attCourse, emoCourse, memCourse} in time synchronization to generate a brain cognitive state report.

Images