A learning effect prediction method based on electroencephalogram and near-infrared dual-mode signals

Abstract

The application discloses a learning effect prediction method based on electroencephalogram near-infrared dual-mode signals and relates to the technical field of information, and solves the problems that evaluation lacks comprehensive supervision, evaluation standards are single, and learning conditions of students cannot be accurately reflected in the prior art. The method comprises the following steps: obtaining basic data, and pre-processing the basic data to obtain pre-processed data; using a training-completed performance evaluation model to perform performance evaluation on the pre-processed data to obtain evaluation performance; and providing learning suggestions according to the evaluation performance. The method realizes space-time-spectrum information fusion and feature extraction, and accurately reflects the learning conditions of students.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a learning effect prediction method based on electroencephalogram-near infrared dual-mode signals. BACKGROUND

[0002] Emerging forms of education bring new challenges. For example, teachers shift from traditional face-to-face teaching to blended learning, which makes it difficult to monitor students' learning state and effect. In addition, students' self-discipline learning ability also becomes a challenge, and they need to adapt to new learning methods and self-manage learning progress.

[0003] Currently, the evaluation of online learning mainly relies on simple learning behavior data such as students' video watching time and test scores. However, this evaluation lacks comprehensive supervision, the evaluation standard is single, and cannot accurately reflect the learning situation of students; the data type is single, the mining degree is limited, and it is difficult to effectively supervise and evaluate students; the evaluation is usually formed after the learning process is over, which is lagging behind, and it is difficult to realize real-time evaluation and early intervention in the learning process. SUMMARY

[0004] The present application provides a learning effect prediction method based on electroencephalogram-near infrared dual-mode signals, which solves the problem of lack of comprehensive supervision and single evaluation standard in the prior art, and realizes space-time-frequency information fusion to extract features and accurately reflect the learning situation of students.

[0005] In a first aspect, the present application provides a learning effect prediction method based on electroencephalogram-near infrared dual-mode signals, which comprises:

[0006] Obtaining basic data and preprocessing the basic data to obtain preprocessed data;

[0007] Using a completed training performance evaluation model to evaluate the preprocessed data to obtain an evaluation score; wherein the performance evaluation model comprises an EEG data processing branch, an fNIRS data processing branch, an information interaction module and a performance regression module; wherein the EEG data processing branch and the fNIRS data processing branch are independent of each other; the EEG data processing branch and the fNIRS data processing branch interact with the information interaction module; the performance regression module obtains data from the EEG data processing branch and the fNIRS data processing branch for performance evaluation;

[0008] According to the evaluation score, learning suggestions are proposed.

[0009] In combination with the first aspect, in a possible implementation manner, the preprocessing of the basic data to obtain preprocessed data comprises:

[0010] filtering the EEG sampling data according to the reference points to obtain re-referenced EEG sampling data;

[0011] filtering the EEG sampling data according to the reference points to obtain re-referenced EEG sampling data;

[0012] band-pass filtering the re-referenced EEG sampling data by using a band-pass filter to obtain preprocessed EEG sampling data;

[0013] removing motion artifacts from the fNIRS sampling data, and band-pass filtering the fNIRS sampling data from which the motion artifacts are removed by using a band-pass filter to obtain preprocessed fNIRS sampling data;

[0014] segmenting the preprocessed EEG sampling data and the preprocessed fNIRS sampling data respectively, and removing a baseline from the segmented EEG sampling data and the segmented fNIRS sampling data respectively to obtain preprocessed data.

[0015] In a possible implementation manner of the first aspect, before the preprocessing of the basic data, the method further includes: screening the basic data by using a questionnaire.

[0016] In a possible implementation manner of the first aspect, the EEG data processing branch includes: a first single-layer convolutional encoder-decoder, a first convolutional multi-head self-attention layer, and a Fourier transform post-computation multi-head self-attention layer connected in sequence.

[0017] The fNIRS data processing branch includes: a second single-layer convolutional encoder-decoder, a second convolutional multi-head self-attention layer, and a third convolutional multi-head self-attention layer connected in sequence.

[0018] The information interaction module includes: a first information interaction layer and a second information interaction layer independent of each other; wherein the first information interaction layer interacts with the EEG data processing branch and the fNIRS data processing branch, and the second information interaction layer interacts with the EEG data processing branch and the fNIRS data processing branch.

[0019] The performance regression module includes: a multi-layer perception layer.

[0020] In a possible implementation manner of the first aspect,

[0021] The first single-layer convolutional encoder-decoder is configured to obtain first reconstructed EEG data according to the preprocessed data.

[0022] The first convolutional multi-head self-attention layer is used to extract the first neural electrical data features from the first reconstructed EEG data;

[0023] The second single-layer convolutional encoder-decoder is used to obtain the second reconstructed fNIRS data based on the preprocessed data;

[0024] The second convolutional multi-head self-attention layer is used to extract the first blood flow data features from the second reconstructed fNIRS data;

[0025] The first information interaction layer is used to calculate the weight of each feature in the first neural electrical data feature based on the first blood flow data feature and the first neural electrical data feature, and use the weight to weight the first neural electrical data feature to obtain first weighted feature data, and calculate the weight of each feature in the first blood flow data feature, and use the weight to weight the first blood flow data feature to obtain second weighted feature data.

[0026] The multi-head self-attention layer is calculated after Fourier transform and used to transform the first weighted feature data to obtain the second neural electrical data feature.

[0027] The third convolutional multi-head self-attention layer is used to extract features from the second weighted feature data to obtain the second blood flow data features;

[0028] The multilayer sensing layer is used to obtain an evaluation score based on the second neural electrical data features and the second blood flow data features.

[0029] In conjunction with the first aspect, in one possible implementation, the loss function used when training the performance evaluation model is expressed as:

[0030] loss = W id loss id +W grade loss grade ;

[0031] Among them, W id Represent the trainable weights to determine the impact of identity token loss on the loss; loss id The identity token loss here represents a classification cross-loss entropy; W grade The trainable weights represent the impact of the performance regression loss on the overall loss; loss grade The Huber loss represents the loss function for performance regression.

[0032] Secondly, the present invention provides a learning effect prediction device based on EEG near-infrared dual-modal signals, the device comprising:

[0033] A preprocessing module is configured to preprocess the basic data to obtain preprocessed data.

[0034] An evaluation module is configured to evaluate the preprocessed data by using a trained achievement evaluation model to obtain an evaluation achievement.

[0035] An output module is configured to provide learning suggestions according to the evaluation achievement and output the learning suggestions.

[0036] In a third aspect, the present application provides a learning effect prediction server based on electroencephalogram and near-infrared dual-mode signals, which comprises a memory and a processor.

[0037] The memory is configured to store computer executable instructions.

[0038] The processor is configured to execute the computer executable instructions to realize a learning effect prediction method based on electroencephalogram and near-infrared dual-mode signals.

[0039] The one or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0040] The present application uses a trained achievement evaluation model to replace the time-consuming signal preprocessing part, thereby improving the real-time performance of the overall method, and then uses the achievement evaluation model to extract the time-space frequency spectrum features of the electroencephalogram signal and the time-space features of the near-infrared signal, effectively solving the problems of high time cost and inaccurate prediction, and further realizing better use of the time regularity existing in data collection, which helps to improve the prediction accuracy of the network. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A learning effect prediction method based on electroencephalogram and near-infrared dual-mode signals is provided for the embodiments of the present application.

[0042] Figure 2 A data preprocessing schematic diagram is provided for the embodiments of the present application.

[0043] Figure 3 An achievement evaluation model structure schematic diagram is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0044] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0045] The present application provides a learning effect prediction method based on electroencephalogram near-infrared dual-mode signals, as shown in the formula (I), the method comprises the following steps S101-S103. Figure 1

[0046] S101, obtain basic data, and pre-process the basic data to obtain pre-processed data.

[0047] Specifically, in step S101, the basic data is pre-processed to obtain pre-processed data, as shown in the formula (II), which comprises steps S1011-S1015. Figure 2

[0048] S1011, screening and classifying the basic data to obtain EEG sampling data and fNIRS sampling data.

[0049] S1012, re-screening reference points for the EEG sampling data, and re-referencing the EEG sampling data according to the reference points to obtain re-referenced EEG sampling data.

[0050] S1013, band-pass filtering the re-referenced EEG sampling data using a band-pass filter to obtain pre-processed EEG sampling data.

[0051] S1014, removing motion artifacts from the fNIRS sampling data, and band-pass filtering the fNIRS sampling data without motion artifacts using a band-pass filter to obtain pre-processed fNIRS sampling data.

[0052] S1015, segmenting the pre-processed EEG sampling data and the pre-processed fNIRS sampling data respectively, and removing baseline from the segmented EEG sampling data and the segmented fNIRS sampling data respectively to obtain pre-processed data.

[0053] Specifically, before pre-processing the basic data, the method further comprises: screening the basic data by questionnaire survey.

[0054] For example, design an EEG-fNIRS experiment and analyze experimental data; including designing an fNIRS-EEG experiment, processing and analyzing experimental data. Design pre-test and post-test questionnaire, and construct a deep model. Among them:

[0055] ​​EEG-fNIRS experimental design: To obtain the neural and hemodynamic responses of students without prior knowledge in learning multidisciplinary knowledge, we first selected two subjects from four academic types according to Biglan's classification to screen learning videos. Then, we designed a pre-test questionnaire to be completed by students before the experiment to ensure that they had no prior knowledge of the subjects to be learned. We edited the learning videos into short videos of about 3 minutes, each containing several knowledge points, and designed corresponding learning questions. We integrated the learning videos and questions into a program through Psychtoolbox, and students participating in the experiment needed to wear EEG and NIRS devices throughout the experiment, following the program's requirements for rest, learning, and answering questions. Each experiment included 4 subjects, 8 video learning contents, and 80 questions, and the entire experiment process took about 45 minutes. The experimental program recorded the students' choices for each question, while the EEG and NIRS devices recorded the neural and hemodynamic data during the experiment.

[0056] EEG experimental data processing: The EEG data obtained in the experiment needs to be preprocessed using Python, including re-reference, band-pass filtering, data segmentation, and baseline removal. The preprocessing of EEG data often involves ICA, which is time-consuming. To improve the real-time efficiency of the model, we removed this step and used a deep learning preprocessing module to replace it in the next step.

[0057] fNIRS experimental data processing: The near-infrared data obtained in the experiment needs to be preprocessed using Python, including motion artifact removal, band-pass filtering, and baseline removal. We also constructed a deep learning preprocessing module in the deep model for further preprocessing.

[0058] Pre-test and post-test questionnaire content: To ensure that the questions are clear and concise, and to consider the time cost of the learners, we designed a simple questionnaire. The questionnaire includes the basic information of the students, such as age, gender, and educational background, and further asks whether the learners have learned or understood the knowledge related to the course, in order to evaluate their prior knowledge level. Then, we use the key concepts, basic principles, and related topics learned before to test the learners' understanding of the relevant concepts before starting to learn. Through the questionnaire results, we excluded students with too much knowledge of the subject to avoid the situation where prior knowledge leads to inconsistent performance and neural and hemodynamic signals. The post-test questionnaire covers the knowledge points in the learning video and their applications to examine the students' understanding of the learning video.

[0059] Step S101 is based on the EEG-fNIRS data of the student himself. Compared with the current learning evaluation method which only relies on online behavior data of a simple dimension or only uses neural information for score prediction, the method is more diversified and personalized. It can detect the brain response of the student, evaluate the learning effect, predict the learning score, and provide reasonable learning suggestions. In addition, the method can also analyze the brain region response in complex tasks, which is of great significance for in-depth understanding of brain mechanism. The method of evaluating prior knowledge by questionnaire before the course starts provides a basis for students to learn new knowledge points. During the learning process, the learning effect is predicted through neural oxygen data sources, and stage learning suggestions are put forward. In addition, the present application can also provide new insights for in-depth understanding of brain mechanism.

[0060] An EEG-fNIRS experimental design is adopted, in which video materials are used. Compared with traditional simple paradigm experiments, this design presents audio and pictures at the same time, which can better simulate real learning scenes and improve the applicability of education evaluation.

[0061] In step S102, the preprocessed data is evaluated by the trained score evaluation model to obtain the evaluation score. The score regression module obtains data from the EEG data processing branch and the fNIRS data processing branch for score evaluation.

[0062] Specifically, in step S102, the score evaluation model, as shown in Figure 3 includes: an EEG data processing branch, an fNIRS data processing branch, an information interaction module, and a score regression module; wherein the EEG data processing branch and the fNIRS data processing branch are independent of each other; the EEG data processing branch and the fNIRS data processing branch interact with the information interaction module.

[0063] The EEG data processing branch includes: a first single-layer convolutional encoder-decoder, a first convolutional multi-head self-attention layer, and a Fourier transform post-computation multi-head self-attention layer connected in sequence.

[0064] The fNIRS data processing branch includes: a second single-layer convolutional encoder-decoder, a second convolutional multi-head self-attention layer, and a third convolutional multi-head self-attention layer connected in sequence.

[0065] The information interaction module includes: a first information interaction layer and a second information interaction layer independent of each other;

[0066] The score regression module includes: a multi-layer perception layer.

[0067] Specifically, in step S102, the functions of each branch point of the score evaluation model include:

[0068] (1) a first single-layer convolutional encoder-decoder for obtaining first reconstructed EEG data from the preprocessed data.

[0069] (2) a first convolutional multi-head self-attention layer for extracting first neuroelectric data features from the first reconstructed EEG data.

[0070] (3) a second single-layer convolutional encoder-decoder for obtaining second reconstructed fNIRS data from the preprocessed data.

[0071] (4) a second convolutional multi-head self-attention layer for extracting first blood flow data features from the second reconstructed fNIRS data.

[0072] (5) a first information interaction layer for calculating weights of each feature in the first neuroelectric data features based on the first blood flow data features and the first neuroelectric data features, weighting the first neuroelectric data features using the weights to obtain first weighted feature data, and calculating weights of each feature in the first blood flow data features, weighting the first blood flow data features using the weights to obtain second weighted feature data.

[0073] (6) a Fourier transform post-computation multi-head self-attention layer for converting the first weighted feature data to obtain second neuroelectric data features. Here, the Fourier transform post-computation multi-head self-attention layer first converts the input data to the frequency domain through Fourier transform. The purpose of this step is to convert the data from the time domain or spatial domain to the frequency domain, so as to better capture the periodicity or frequency characteristics in the data, and then apply the multi-head self-attention mechanism to process the transformed data. The multi-head self-attention mechanism calculates the relationship between different parts of the input data through multiple attention heads (multiple self-attention sub-layers) in parallel. The purpose of this step is to capture important information in the frequency domain features through the mechanism of the multi-head self-attention layer, and combine the results of multiple attention heads to generate the final feature representation.

[0074] (7) a third convolutional multi-head self-attention layer for extracting features from the second weighted feature data to obtain second blood flow data features.

[0075] (8) a multi-layer perception layer for obtaining an evaluation result based on the second neuroelectric data features and the second blood flow data features.

[0076] For example, the encoder part of the first single-layer convolutional encoder-decoder and the second single-layer convolutional encoder-decoder is responsible for converting the input data into a representation in the latent space, which contains the key features of the input data and can better process and analyze the data in the latent space.

[0077] The decoder part of the first and second single-layer convolutional encoder-decoder is responsible for decoding the encoded representation in the latent space into reconstructed data in the original data space. The decoder is also usually a deep neural network with a structure opposite to the encoder, gradually decoding the encoded representation in the latent space into reconstructed data similar to the original input data through a series of layers. The reconstructed data generated by the decoder has similar structure and features to the original input data, but may have slight differences. These differences are considered as noise in the original data.

[0078] Due to the different characteristics of the preprocessed data (EEG-fNIRS), different feature extraction modules need to be designed according to their characteristics.

[0079] Since the electroencephalogram signal has a wide frequency spectrum, from low frequency (such as delta wave) to high frequency (such as gamma wave), constructing a deep spectral extraction module helps to extract effective information from it. The first convolutional multi-head self-attention layer is used to extract the features of time domain and space domain from the signal, including the dynamic changes of time series data and the spatial relationship between different channels, to obtain the first neural electrical data features.

[0080] The near-infrared signal has weaker spectral information than the electroencephalogram signal, so it can not need a deep spectral extraction module, but only a second convolutional multi-head self-attention layer to capture the time sequence and spatial information in the signal. The second convolutional multi-head self-attention layer is used to capture the time sequence and spatial information in the signal to obtain the first blood flow data features.

[0081] The first neural electrical data features and the first blood flow data features are interacted through the first information interaction layer during the feature extraction process. The specific interaction process includes:

[0082] The first neural electrical data features and the first blood flow data features are processed through their respective neural networks in parallel, and various features are continuously extracted therefrom. However, the data of different modalities have complementary information, and the electroencephalogram signal and the near-infrared signal can provide different aspects of brain activity information. The information interaction module can promote the fusion of cross-modal information and make full use of the information of the two modalities to improve the overall feature representation ability.

[0083] Specifically, in step S102, the loss function used when training the performance evaluation model is represented as:

[0084] loss=W id ·loss id +W grade ·loss grade ;

[0085] Wherein, W id represents a trainable weight to determine the influence of the identity token loss on loss; lossid represents the identity token loss here is a categorical cross-entropy loss; W grade represents the trainable weights to determine the impact of the achievement regression loss on loss; loss grade represents the loss function of the achievement regression Huber loss.

[0086] In step S102, the real-time performance of data preprocessing is improved, simple preprocessing is first performed, and then an encoder and a decoder are constructed for further preprocessing.

[0087] In step S103, learning suggestions are proposed according to the evaluation achievements.

[0088] For example, the behavior data and the personalized portrait are combined to evaluate and predict: the personalized portrait can intuitively describe the learning characteristics and learning feelings of a student, and can reflect the learning ability of the student to a certain extent. The behavior data records the daily learning track of the student, and the learning achievement predicted from the behavior data can reflect the performance and the input degree of the student when learning a course. If the behavior data of a student predicts a low achievement, and the personal portrait shows that the student has good learning ability, the student can be reminded to input more effort on the course.

[0089] In step S103, the student neural blood flow information is combined to predict the achievement, and a personalized evaluation result is given, so that appropriate learning suggestions can be proposed for the student, and the learning enthusiasm of the student is promoted, the student actively learns, and the learning effectiveness is improved.

[0090] The application further provides a learning effect prediction device based on electroencephalogram near-infrared dual-mode signals, which comprises a preprocessing module, an evaluation module and an output module.

[0091] The preprocessing module is used for preprocessing the basic data to obtain preprocessed data.

[0092] The evaluation module is used for performing achievement evaluation on the preprocessed data by using the trained achievement evaluation model to obtain evaluation achievements.

[0093] The output module is used for proposing learning suggestions according to the evaluation achievements and outputting the suggestions.

[0094] Some of the modules in the apparatus of the present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0095] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary hardware. Based on such an understanding, the technical solutions of the present application, which are essential or contribute to the prior art, can be embodied in the form of a software product or in the form of a data migration implementation process. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the method of each embodiment or some part of the embodiments of the present application.

[0096] The present application provides a learning effect prediction server based on electroencephalogram and near-infrared dual-mode signals, which comprises a memory and a processor; the memory is used for storing computer executable instructions;

[0097] The processor is used for executing the computer executable instructions to realize a learning effect prediction method based on electroencephalogram and near-infrared dual-mode signals.

[0098] Although the present application provides method operation steps such as embodiments or flowcharts, more or fewer operation steps can be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is only one of the many execution orders, and does not represent the only execution order. In actual device or client product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-thread processing environment).

[0099] The apparatus or modules illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above apparatus is described as various modules respectively described in terms of functions. In the implementation of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the modules implementing certain functions can also be implemented by a combination of multiple sub-modules or sub-units.

[0100] The above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some or all of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A learning effect prediction method based on electroencephalogram and near-infrared dual-modal signals, characterized in that, The method comprises: obtaining basic data and preprocessing the basic data to obtain preprocessed data; The preprocessing of the basic data to obtain preprocessed data comprises: screening and classifying the basic data to obtain EEG sample data and fNIRS sample data; re-screening reference points of the EEG sample data and re-referencing the EEG sample data according to the reference points to obtain re-referenced EEG sample data; band-pass filtering the re-referenced EEG sample data using a band-pass filter to obtain preprocessed EEG sample data; removing motion artifacts from the fNIRS sample data and band-pass filtering the fNIRS sample data from which the motion artifacts have been removed using a band-pass filter to obtain preprocessed fNIRS sample data; segmenting the preprocessed EEG sample data and the preprocessed fNIRS sample data respectively, and removing the baseline of the segmented EEG sample data and the segmented fNIRS sample data respectively to obtain preprocessed data; using the trained performance evaluation model to evaluate the preprocessed data to obtain an evaluation performance; wherein the performance evaluation model comprises an EEG data processing branch, an fNIRS data processing branch, an information interaction module and a performance regression module; wherein the EEG data processing branch and the fNIRS data processing branch are independent of each other; the EEG data processing branch and the fNIRS data processing branch interact with the information interaction module; the performance regression module obtains data from the EEG data processing branch and the fNIRS data processing branch for performance evaluation; the EEG data processing branch comprises a first single-layer convolutional encoder-decoder, a first convolutional multi-head self-attention layer and a Fourier transform calculation multi-head self-attention layer connected in sequence; the fNIRS data processing branch comprises a second single-layer convolutional encoder-decoder, a second convolutional multi-head self-attention layer and a third convolutional multi-head self-attention layer connected in sequence; the information interaction module comprises a first information interaction layer and a second information interaction layer independent of each other; wherein the first information interaction layer interacts with the EEG data processing branch and the fNIRS data processing branch, and the second information interaction layer interacts with the EEG data processing branch and the fNIRS data processing branch; the performance regression module comprises a multi-layer perception layer; According to the evaluation performance, learning suggestions are proposed. 2.The learning effect prediction method based on electroencephalogram and near-infrared dual-modal signals according to claim 1, characterized in that, Before preprocessing the basic data, the method further comprises: screening the basic data by questionnaire survey.

3. The learning effect prediction method based on electroencephalogram and near-infrared spectroscopy dual-modal signals according to claim 1, characterized in that: The first single-layer convolutional encoder-decoder is configured to obtain first reconstructed EEG data from the preprocessed data; The first convolutional multi-head self-attention layer is configured to extract first neural electric data features from the first reconstructed EEG data; The second single-layer convolutional encoder-decoder is configured to obtain second reconstructed fNIRS data from the preprocessed data; The second convolutional multi-head self-attention layer is configured to extract first blood flow data features in the second reconstructed fNIRS data. The first information interaction layer is configured to calculate weights of respective features in the first neuroelectric data features according to the first blood flow data features and the first neuroelectric data features, to obtain first weighted feature data by weighting the first neuroelectric data features using the weights, and to calculate weights of respective features in the first blood flow data features, to obtain second weighted feature data by weighting the first blood flow data features using the weights. The post-Fourier transform and multi-head self-attention layer is configured to convert the first weighted feature data to obtain second neuroelectric data features. The third convolutional multi-head self-attention layer is configured to extract second blood flow data features from the second weighted feature data. The multi-layer perception layer is configured to obtain an evaluation result according to the second neuroelectric data features and the second blood flow data features. 4.The learning effect prediction method based on electroencephalogram and near-infrared dual-modal signals according to claim 1, characterized in that, A loss function used when training the result evaluation model is represented as: ； wherein, represents trainable weights to determine the impact of the identity token loss on the loss; represents the identity token loss here is a categorical cross-entropy loss with respect to the identity token; represents trainable weights to determine the impact of the performance regression loss on the loss; represents the loss function for the performance regression, a Huber loss. 5.A learning effect prediction device based on electroencephalogram and near-infrared dual-modal signals, characterized in that, The method for predicting learning effect based on electroencephalogram and near-infrared dual-modal signals according to any one of claims 1-4 comprises: a preprocessing module configured to preprocess basic data to obtain preprocessed data; an evaluation module configured to evaluate the preprocessed data using the trained result evaluation model to obtain an evaluation result; an output module configured to output learning suggestions according to the evaluation result. 6.A learning effect prediction server based on electroencephalogram and near-infrared dual-modal signals, characterized in that, comprising a memory and a processor; the memory is configured to store computer executable instructions; the processor is configured to execute the computer executable instructions to implement the method for predicting learning effect based on electroencephalogram and near-infrared dual-modal signals according to any one of claims 1-4.

Images