A method, device and system for recognizing a state of a high-order cognitive activity of a learner

Abstract

The application provides a learner high-order cognitive activity state recognition method, device and system, and determines a hierarchical convolution capsule network model; the hierarchical convolution capsule network model increases the length of the convolution kernel of all convolution operation channel directions to a preset value on the basis of an original capsule network model, so that the receptive field of each convolution operation can effectively capture the correlation characteristics of the corresponding electroencephalogram channels of the left hemisphere and the right hemisphere of the brain of the learner in the high-order cognitive state and the low-order cognitive state, to extract effective information representing the high-order cognitive activity state of the learner; and the hierarchical convolution capsule network increases two convolution layers in the convolution activation module of the original capsule network model, to construct a hierarchical convolution module, to extract more abundant features of different levels of electroencephalogram data, to further effectively capture information representing the high-order cognitive activity state of the learner; and the electroencephalogram data of the learner in the learning process is input into the trained hierarchical convolution capsule network model, to perceive whether the learner is in the high-order cognitive activity state.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition of EEG using deep learning models, and more specifically, relates to a method, device and system for recognizing the higher-order cognitive activity states of learners. Background Technology

[0002] Higher-order thinking skills are essential for students to face future challenges, and cultivating these skills is a crucial topic of ongoing exploration in education both domestically and internationally. Research on evaluating students' higher-order thinking abilities has garnered significant attention in the teaching process. Existing assessments of higher-order thinking skills primarily focus on learning outcomes, requiring the design of appropriate test questions for specific subjects, followed by subjective analysis of the results. These methods are susceptible to the subjectivity of the evaluator, and the effectiveness of measuring higher-order thinking is highly correlated with the quality of test design. Furthermore, feedback is often delayed, resulting in low measurement efficiency and unstable results. Therefore, in the teaching process, timely, efficient, and objectively assessing whether students are engaging in higher-order thinking and evaluating their cognitive activity is crucial for better cultivating their higher-order thinking abilities.

[0003] Electroencephalography (EEG), as a physiological signal, has been widely used in task recognition across various fields, such as emotion recognition and sleep stage classification. Therefore, current models for automatically classifying EEG data using artificial intelligence algorithms are mostly constructed in conjunction with specific task characteristics; no data has yet been found using EEG to study the classification of students' high- and low-order cognitive activities. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method, device and system for recognizing the higher-order cognitive activity state of learners, aiming to solve the problem that the prior art cannot timely, efficiently and objectively perceive whether students are in a higher-order cognitive state.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for recognizing higher-order cognitive activity states of learners, comprising the following steps:

[0006] determine a hierarchical convolution capsule network model; the hierarchical convolution capsule network model is used to perceive a cognitive state of a learner based on electroencephalogram data in a learning activity process of the learner, and determine whether the learner is in a high-order cognitive state; the hierarchical convolution capsule network model increases a length of a convolution kernel in all convolution operation channel directions to a preset value based on an original capsule network model, so that a receptive field of each convolution operation can effectively capture correlation features of corresponding electroencephalogram channels of a left hemisphere and a right hemisphere of a brain of the learner in two states of high-order and low-order cognitive activities, to extract effective information representing a high-order cognitive activity state of the learner; and the hierarchical convolution capsule network increases two convolution layers in a convolution activation module of the original capsule network model, to construct a hierarchical convolution module, to extract more abundant features of different levels of the electroencephalogram data, to further effectively capture information representing the high-order cognitive activity state of the learner.

[0007] input the electroencephalogram data in the learning activity process of the learner into the trained hierarchical convolution capsule network model, to perceive whether the learner is in the high-order cognitive activity state when performing the learning activity.

[0008] In an optional example, the method further includes the following steps:

[0009] determine electroencephalogram data of the learner in different types of learning activities as training data; the learning activities include: copying learning materials, watching learning videos, and constructing concept maps; when the learner copies the learning materials and watches the learning videos, the learner is in a low-order cognitive state, and when the learner constructs the concept maps, the learner is in a high-order cognitive state;

[0010] input the training data into the improved capsule network model, train the model, and obtain the trained hierarchical convolution capsule network model.

[0011] In an optional example, the preset value is 17, that is, the length of the convolution kernel is increased to 17, and the width remains unchanged.

[0012] Specifically, when the length of the convolution kernel is 17, each convolution can cover the left and right hemispheres of the brain at the same time. If the length of the convolution kernel is greater than 17, the number of convolutions will be reduced, which may have a bad effect on extracting fine-grained features. If the length of the convolution kernel is less than 17, each convolution cannot cover the left and right hemispheres of the brain at the same time.

[0013] In an optional example, two convolution layers are added after one convolution layer of the convolution activation module, the two added convolution layers also use a convolution kernel with a length of 17, the convolution step length is consistent with that of the first convolution layer, and a padding strategy is used during convolution to ensure that the size of the output feature map is the same as that of the original convolution activation module output by the first convolution.

[0014] In an optional example, the original capsule network model is a multi-feature guided capsule network model.

[0015] In a second aspect, the present application provides a learner high-order cognitive activity state recognition device, comprising:

[0016] a model determination unit configured to determine a hierarchical convolution capsule network model; the hierarchical convolution capsule network model is configured to perceive a cognitive state of a learner based on electroencephalogram data in a learning activity process of the learner, and determine whether the learner is in a high-order cognitive state; the hierarchical convolution capsule network model increases a length of a convolution kernel in all convolution operation channel directions to a preset value based on an original capsule network model, so that a receptive field of each convolution operation can effectively capture correlation features of corresponding electroencephalogram channels of a left hemisphere and a right hemisphere of a brain of the learner in a high-order cognitive state and a low-order cognitive state, to extract effective information representing a high-order cognitive activity state of the learner; and the hierarchical convolution capsule network adds two convolution layers to a convolution activation module of the original capsule network model, to construct a hierarchical convolution module, to extract more abundant features of different levels of the electroencephalogram data, to further effectively capture information representing the high-order cognitive activity state of the learner.

[0017] a cognitive state recognition unit configured to input the electroencephalogram data in the learning activity process of the learner to the trained hierarchical convolution capsule network model, to perceive whether the learner is in the high-order cognitive activity state when performing the learning activity.

[0018] In an optional example, the device further comprises:

[0019] a model training unit configured to determine electroencephalogram data of the learner in different types of learning activities, as training data; the learning activities include: copying learning materials, watching learning videos, and constructing concept maps; when the learner is copying the learning materials and watching the learning videos, the learner is in a low-order cognitive state, and when the learner is constructing the concept maps, the learner is in a high-order cognitive state; and inputting the training data to the improved capsule network model, training the model, and obtaining the trained hierarchical convolution capsule network model.

[0020] In an optional example, the preset value is 17, that is, the length of the convolution kernel is increased to 17, and the width remains unchanged.

[0021] In a third aspect, the present application provides a learner high-order cognitive activity state recognition system, comprising: a memory and a processor.

[0022] The memory is configured to store a computer program.

[0023] The processor is configured to implement the method provided in the first aspect when executing the computer program.

[0024] In a fourth aspect, the present application provides a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method provided in the first aspect above is implemented.

[0025] Overall, compared with the prior art, the above technical solutions conceived by the present application have the following beneficial effects:

[0026] The present application provides a learner high-order cognitive activity state recognition method, device and system, which acquires the brain electrical data of students in a companion mode, pre-processes the original data to form a brain electrical data set, and finally improves the algorithm based on the capsule network model in the field of artificial intelligence, so as to automatically capture the correlation characteristics of the corresponding brain electrical channels of the left and right hemispheres of the brain, and automatically extract deeper features of the brain electrical data, and distinguish the high-order and low-order cognitive activity states of students. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a learner high-order cognitive activity state recognition method flowchart provided by an embodiment of the present application;

[0028] Figure 2 is another intelligent method flowchart for perceiving the high-order cognitive activity state of a learner provided by an embodiment of the present application;

[0029] Figure 3 is a learner high-order cognitive activity state recognition device architecture diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0031] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0032] The present application first utilizes the objective and unforgeable advantage of the brain electrical data to reflect the high-order cognitive activity state of a person externally, and then introduces an artificial intelligence algorithm to classify the brain electrical data, so as to realize efficient and timely perception of the high-order cognitive activity state of a learner.

[0033] The application discloses a learner high-order cognitive activity state recognition method, device and system, and is based on the participation of students in online high-order and low-order learning activities, and brain electrical data of the students is collected in a companion mode, and after the original data is preprocessed, a brain electrical data set is formed, and finally, an algorithm is improved based on a capsule network model in the field of artificial intelligence, so that the correlation characteristics of corresponding brain electrical channels of the left hemisphere and the right hemisphere of the brain are automatically captured, and deeper characteristics of the brain electrical data are automatically extracted, and the high-order and low-order cognitive activity states of the students are distinguished.

[0034] Figure 1 It is a learner high-order cognitive activity state recognition method flowchart provided by the embodiment of the application; as shown in the figure, Figure 1 The method comprises the following steps:

[0035] S101, determine a hierarchical convolution capsule network model; the hierarchical convolution capsule network model is used for perceiving the cognitive state of a learner based on brain electrical data in the learning activity process of the learner, and judging whether the learner is in a high-order cognitive state; the hierarchical convolution capsule network model increases the length of the convolution kernel of all convolution operation channel directions to a preset value on the basis of the original capsule network model, so that the receptive field of each convolution operation can effectively capture the correlation characteristics of corresponding brain electrical channels of the left hemisphere and the right hemisphere of the learner in the high-order and low-order cognitive states, so as to extract effective information representing the high-order cognitive activity state of the learner; and the hierarchical convolution capsule network increases two convolution layers in the convolution activation module of the original capsule network model, and constructs a hierarchical convolution module, so as to extract more rich characteristics of the brain electrical data at different levels, and further effectively capture information representing the high-order cognitive activity state of the learner;

[0036] S102, input the brain electrical data of the learner in the learning activity process into the trained hierarchical convolution capsule network model, so as to perceive whether the learner is in a high-order cognitive activity state when learning.

[0037] In an optional example, the method further comprises the following steps:

[0038] The brain electrical data of the learner in different types of learning activities is determined as training data; the learning activities include: copying learning materials, watching learning videos and constructing concept maps; when the learner copies learning materials and watches learning videos, the learner is in a low-order cognitive state, and when the learner constructs concept maps, the learner is in a high-order cognitive state;

[0039] The training data is input into the improved capsule network model, the model is trained, and the trained hierarchical convolution capsule network model is obtained.

[0040] In an optional example, the preset value is 17, that is, the length of the convolution kernel is increased to 17, and the width remains unchanged.

[0041] In an optional example, two convolutional layers are added after a convolutional layer of the convolutional activation module, the two added convolutional layers also use a convolution kernel with a length of 17, the convolution step length is consistent with the first layer convolution, and a padding strategy is used during convolution to ensure that the size of the output feature map is the same as that of the feature map output by the first layer convolution.

[0042] In an optional example, the original capsule network model is a multi-feature guided capsule network model.

[0043] Specifically, according to the two educational theories of ICAP framework and Bloom taxonomy, the present application designs online high-order and low-order learning activities, and collects the electroencephalogram data of students in a companion mode, explores the electroencephalogram reflecting the high-order cognitive activity state of students objectively, and provides objective data support for teaching decision-making. The capsule network model in deep learning is used to classify the electroencephalogram data of students in the process of high-order and low-order learning activities, perceive the high-order cognitive development state of students, and provide efficient support of intelligent algorithm for teachers and students to optimize the teaching and learning strategies in the teaching process in time.

[0044] The present application improves the capsule network model from two aspects, including capturing the internal correlation between the electrodes of the left and right hemispheres of the brain corresponding to the electroencephalogram in the high-order cognitive state, so that the model can more effectively extract the correlation features of the corresponding brain electrical channels of the left and right hemispheres of the brain; and deepening the model depth to extract more rich features of the brain electrical data at different levels. Thus, the accuracy of high-order cognitive state discrimination is improved. Specifically, first, the human brain is structurally divided into left and right hemispheres, and cooperates to play the role of the whole central system. Therefore, when the brain is in a high-order cognitive state, there is an internal relationship between each channel, which represents the cognitive state of the person in correlation, and the left and right hemispheres are connected to each other and support the activity of the brain together. In order to more effectively capture the internal correlation between each channel (electrode) of the EEG related to the cognitive state, and to cover the electrodes of the left and right hemispheres at the same time in each convolution operation, the length of the convolution kernel in the channel direction of all convolution operations in the whole model is increased to 17, and when the convolution kernel with a length of 17 is used to do convolution operation along the channel direction, each convolution simultaneously covers the channels of the left and right hemispheres. In this paper, a larger receptive field is used to more effectively capture the correlation features of the corresponding brain electrical channels of the left and right hemispheres in the high-order and low-order cognitive activity state, so that the model can more effectively capture the correlation features between the channels of the brain.

[0045] Secondly, since the EEG signal represents the activity of the human brain in a more delicate way, and the idea of extracting features from fine (local pixels) to coarse (edges, local patterns) is used in analog image feature extraction, this study hopes to deepen the model depth and thus extract more rich features of different levels of EEG, enhancing the feature expression ability of the model. Two layers of convolution are added in the convolution activation module to construct the hierarchical convolution module (HConvs) to extract deeper features of different levels. This module contains three convolution layers, which are responsible for extracting local features from the raw data of each channel of the EEG sample and inputting them to the primary capsule module.

[0046] The input is the raw EEG sample, the height is 32 channels (C = 32), and the width is 128 sampling points for 1 second (L = 128). The first convolution layer (Conv1) has 256 9x17 convolution kernels with a step size of 1 and no padding (no padding), and outputs a 256-channel feature map with a width of W (W = 120) and a height of H (H = 16). This study increases two convolution layers (Conv2 and Conv3), each containing 256 9x17 convolution kernels with a step size of 1 and a padding strategy (padding = 4) to maintain the same width (W = 120) and height (H = 16) as the feature map output by the first layer of convolution. Experimental results show that the two improvements of simultaneously capturing the connection between the left and right hemispheres of the brain and deepening the model depth can more effectively improve the model performance of identifying whether the learner is in a high-order cognitive activity state.

[0047] Figure 2 is another intelligent method flowchart for perceiving the high-order cognitive activity state of a learner provided by an embodiment of the present application, as shown in Figure 2 The method comprises the following steps.

[0048] (1) According to the learning activities listed in the ICAP framework, the learning activity of watching learning videos without doing anything belongs to passive learning, and the cognitive of the learner only reaches the minimum level of understanding; the learning activity of copying learning materials is equivalent to making learning notes word by word, which belongs to active learning, and the cognitive of the learner only reaches the level of shallow understanding; constructing a concept map belongs to a constructive learning activity, and the construction behavior usually requires an inference process, including new construction, modification, reorganization and reflection.

[0049] According to the cognitive categories of Bloom's taxonomy, when students perform the learning activities of copying learning materials and watching learning videos, their cognitive participation is limited to the levels of memory and understanding of knowledge, and the students are in a low-order cognitive activity state; in the learning activity of drawing a concept map, the cognitive process of the students contains reasoning and analysis, and the students are in a high-order cognitive activity state.

[0050] Based on the two educational theories of ICAP framework and Bloom taxonomy, three learning activities are designed in the online learning scenario, which are copying learning materials, watching learning videos and constructing concept maps according to videos. It is considered that when learners participate in low-level learning activities such as copying learning materials and watching learning videos, they are in a low-order cognitive activity state, and whether they are in a high-order cognitive activity state when participating in high-level learning activities such as constructing concept maps.

[0051] (2) EEG data of each student during the three learning activities of copying learning materials, watching learning videos and constructing concept maps according to videos are collected in a companion manner without interruption by using an EEG cap, and a complete set of raw EEG data of each person is obtained.

[0052] The EEG cap used has multiple electrodes, such as 32 electrodes.

[0053] (3) After collecting the EEG data of each student during the entire experiment, the EEG data sample of each student is obtained by labeling, improving the signal-to-noise ratio, and cutting, and then the samples of all students are randomly mixed to construct an EEG data set.

[0054] (4) The raw EEG sample is input into the capsule network in the deep learning algorithm, and the capsule network model is improved from two aspects. First, the intrinsic correlation between the corresponding electrodes of the left and right hemispheres of the brain under high-order cognitive state is captured, so that the model can more effectively extract the correlation features between the left and right hemispheres of the brain; second, the depth of the model is deepened to extract deeper and richer features of the EEG data. Thus, the accuracy of high-order cognitive state discrimination is improved.

[0055] Corresponding to the method embodiment, the application also proposes a system for perceiving the high-order cognitive activity state of students, which comprises:

[0056] (1) Data acquisition module: students complete the learning activities of copying learning materials, watching learning videos and constructing concept maps according to videos on an online platform, and EEG data of the students is collected in a companion manner. Figure Three (2) Data preprocessing module: for the raw EEG data of each student, the start and end times of the EEG of the three learning activities are marked with mark, the spatial positions of the 32 electrodes are positioned, the bandpass filter is used to remove the power frequency interference and save the required frequency range, the three learning activity data segments are cut from the complete EEG data of each student, the independent component analysis is performed to remove the horizontal and vertical electrooculogram interference components. Then, the three data segments of each student are labeled, the EEG samples are cut by using a sliding window, and finally the samples of all students are shuffled to form an EEG data set.

[0057] (2) Data preprocessing module: for the raw EEG data of each student, the start and end times of the EEG of the three learning activities are marked with mark, the spatial positions of the 32 electrodes are positioned, the bandpass filter is used to remove the power frequency interference and save the required frequency range, the three learning activity data segments are cut from the complete EEG data of each student, the independent component analysis is performed to remove the horizontal and vertical electrooculogram interference components. Then, the three data segments of each student are labeled, the EEG samples are cut by using a sliding window, and finally the samples of all students are shuffled to form an EEG data set.

[0058] (3) intelligent perception module: improve the capsule network model to automatically capture the internal correlation of the left and right hemispheres of the brain, and deepen the model depth to extract deeper and richer features of the electroencephalogram, and realize the perception of whether the student is in a high-order cognitive activity state.

[0059] Figure 3 is the learning high-order cognitive activity state recognition device architecture provided by the embodiment of the application, as shown in Figure 3 , comprising:

[0060] The model determination unit 310 is configured to determine a hierarchical convolution capsule network model; the hierarchical convolution capsule network model is configured to perceive the cognitive state of the learner based on the electroencephalogram data in the learning activity process of the learner, and determine whether the learner is in a high-order cognitive state; the hierarchical convolution capsule network model increases the length of the convolution kernel in all convolution operation channel directions to a preset value based on the original capsule network model, so that the receptive field of each convolution operation can effectively capture the correlation features of the corresponding electroencephalogram channels of the left and right hemispheres of the brain of the learner in the high-order and low-order cognitive states, so as to extract effective information representing the high-order cognitive activity state of the learner; and the hierarchical convolution capsule network adds two convolution layers to the convolution activation module of the original capsule network model to construct a hierarchical convolution module, so as to extract richer features of the electroencephalogram data at different levels, and further effectively capture information representing the high-order cognitive activity state of the learner.

[0061] The cognitive state recognition unit 320 is configured to input the electroencephalogram data in the learning activity process of the learner into the trained hierarchical convolution capsule network model, so as to perceive whether the learner is in a high-order cognitive activity state when performing the learning activity.

[0062] The model training unit 330 is configured to determine the electroencephalogram data under different types of learning activities of the learner, and use the electroencephalogram data as training data; the learning activities include: copying learning materials, watching learning videos, and constructing concept maps; when the learner is copying learning materials and watching learning videos, the learner is in a low-order cognitive state, and when the learner is constructing concept maps, the learner is in a high-order cognitive state; and the training data is input into the improved capsule network model to train the model, and a trained hierarchical convolution capsule network model is obtained.

[0063] It can be understood that the detailed function implementation of each unit can be referred to the description in the foregoing method embodiment, which will not be described here.

[0064] In addition, the embodiment of the application provides another learning high-order cognitive activity state recognition system, which comprises a memory and a processor.

[0065] The memory is configured to store a computer program.

[0066] The processor is configured to implement the method in the above embodiments when executing the computer program.

[0067] The application further provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method in the above embodiments is implemented.

[0068] Based on the method in the above embodiments, the application provides a computer program product. When the computer program product is run on a processor, the processor executes the method in the above embodiments.

[0069] Based on the method in the above embodiments, the application further provides a chip, which comprises one or more processors and an interface circuit. Optionally, the chip can further comprise a bus. Wherein:

[0070] The processor can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor. The processor can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method and step disclosed in the embodiments of the application can be implemented or executed. The general processor can be a microprocessor or any conventional processor.

[0071] The interface circuit can be used for sending or receiving data, instructions or information. The processor can process the data, instructions or other information received by the interface circuit, and can send the processed information through the interface circuit.

[0072] Optionally, the chip further comprises a memory, which can include a read-only memory and a random access memory, and provides operation instructions and data for the processor. A part of the memory can further include a non-volatile random access memory (NVRAM).

[0073] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling operation instructions stored in the memory (the operation instructions can be stored in an operating system).

[0074] Optionally, the interface circuit can be used for outputting the execution result of the processor.

[0075] It should be noted that the functions of the processor and the interface circuit correspond to each other, which can be realized by hardware design, software design or a combination of hardware and software, and is not limited here.

[0076] It should be understood that each step of the above method embodiments can be completed by a logic circuit in the form of hardware in the processor or instructions in the form of software.

[0077] It can be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in some possible implementations, each step in the above embodiments can be selectively executed, partially executed or fully executed according to actual conditions, which is not limited here.

[0078] It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0079] The method steps in the embodiments of the present application can be implemented in the form of hardware or by the processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

[0080] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0081] Those skilled in the art will readily understand that the above description is only preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for recognizing a state of a learner's higher-order cognitive activity, characterized by, The method comprises the following steps: determining a hierarchical convolution capsule network model; the hierarchical convolution capsule network model is used for perceiving the cognitive state of a learner based on electroencephalogram data in the learning activity process of the learner, and judging whether the learner is in a high-order cognitive activity state; the hierarchical convolution capsule network model increases the length of the convolution kernel of all convolution operation channel directions to a preset value on the basis of an original capsule network model, so that the receptive field of each convolution operation can effectively capture the correlation characteristics of the corresponding electroencephalogram channels of the left hemisphere and the right hemisphere of the brain of the learner in the two states of high-order and low-order cognition, so as to extract effective information representing the high-order cognitive activity state of the learner; and the hierarchical convolution capsule network increases two convolution layers in the convolution activation module of the original capsule network model to construct a hierarchical convolution module, so as to extract more rich features of different levels of electroencephalogram data, and further effectively capture information representing the high-order cognitive activity state of the learner; inputting the electroencephalogram data in the learning activity process of the learner into the trained hierarchical convolution capsule network model, so as to perceive whether the learner is in a high-order cognitive activity state when performing the learning activity; further comprising the following steps: determining electroencephalogram data of the learner in different types of learning activities as training data; the learning activities include: copying learning materials, watching learning videos and constructing concept maps; when the learner is copying learning materials and watching learning videos, the learner is in a low-order cognitive state, and when the learner is constructing concept maps, the learner is in a high-order cognitive state; inputting the training data into the improved capsule network model to train the model, and obtaining a trained hierarchical convolution capsule network model; the preset value is 17, that is, the length of the convolution kernel is increased to 17, and the width remains unchanged; two convolution layers are added after one convolution layer of the convolution activation module; the two added convolution layers also use a convolution kernel with a length of 17, the convolution step length is consistent with that of the first convolution layer, and a padding strategy is used during convolution to ensure that the size of the output feature map is the same as that of the feature map output by the first convolution layer; the original capsule network model is a multi-feature guided capsule network model.

2. A learner higher-order cognitive activity state recognition device characterized by comprising: comprise: a model determination unit configured to determine a hierarchical convolution capsule network model; the hierarchical convolution capsule network model is used for perceiving the cognitive state of a learner based on electroencephalogram data in the learning activity process of the learner, and judging whether the learner is in a high-order cognitive activity state; the hierarchical convolution capsule network model increases the length of the convolution kernel of all convolution operation channel directions to a preset value on the basis of an original capsule network model, so that the receptive field of each convolution operation can effectively capture the correlation characteristics of the corresponding electroencephalogram channels of the left hemisphere and the right hemisphere of the brain of the learner in the two states of high-order and low-order cognition, so as to extract effective information representing the high-order cognitive activity state of the learner; and the hierarchical convolution capsule network increases two convolution layers in the convolution activation module of the original capsule network model to construct a hierarchical convolution module, so as to extract more rich features of different levels of electroencephalogram data, and further effectively capture information representing the high-order cognitive activity state of the learner; The cognitive state recognition unit is configured to input the electroencephalogram data in the learning activity of the learner into the trained hierarchical convolutional capsule network model to perceive whether the learner is in a high-order cognitive activity state when performing the learning activity. Further comprising: The model training unit is configured to determine electroencephalogram data of the learner in different types of learning activities as training data, wherein the learning activities include: copying learning materials, watching learning videos, and constructing concept maps; the learner is in a low-order cognitive state when copying learning materials and watching learning videos, and the learner is in a high-order cognitive state when constructing concept maps; and the training data is input into the improved capsule network model to train the model, and a trained hierarchical convolutional capsule network model is obtained. The preset value is 17, that is, the length of the convolution kernel is increased to 17, and the width remains unchanged. Two convolution layers are added after a convolution layer of the convolution activation module, the two added convolution layers also use a convolution kernel with a length of 17, the convolution step length is consistent with that of the first convolution layer, and a padding strategy is used during convolution to ensure that the size of the output feature map is the same as that of the feature map output by the first convolution layer. The original capsule network model is a multi-feature guided capsule network model.

3. A system for recognizing a state of a learner's higher-order cognitive activity, characterized by, Comprise: A memory and a processor; The memory is configured to store a computer program; The processor is configured to implement the method of claim 1 when executing the computer program.

4. A computer-readable storage medium, characterized in that, The storage medium has a computer program stored thereon, and when the computer program is executed by the processor, the method of claim 1 is implemented.

Images