Cognitive load evaluation method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a cognitive load evaluation method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: obtaining learning state data and answer condition data of a target user, the learning state data being used for describing the state change of the target user in the process of learning a specified course, and the answer condition data being used for representing the answer condition of the target user; the answer condition data comprises a specified question replied by the target user and reply correctness information; difficulty information is acquired, and the difficulty information is used for representing difficulty levels of the specified courses and the specified questions; processing the learning state data by using a concentration degree evaluation model to obtain concentration degree evaluation information; processing the answer condition data by using a knowledge tracking model to obtain knowledge mastering information; and processing the concentration evaluation information, the knowledge mastering information and the difficulty information by using a cognitive load evaluation model to obtain cognitive load evaluation information. According to the method, scientificity, professionality and objectivity of cognitive load evaluation can be improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of intelligent teaching technology, and more specifically, to a cognitive load assessment method, device, electronic device, storage medium and program product. Background Art

[0002] Cognitive load refers to the sum of mental effort required to perform a task, which was proposed by cognitive psychologist John Sweller in 1988. The study and evaluation of cognitive load can stimulate learners' initiative and enthusiasm. Understanding and managing cognitive load is crucial to optimizing the learning process and improving learning outcomes.

[0003] In the existing technology, VR (Virtual Reality) is widely used in education and training to provide immersive learning experience. However, evaluating learners' cognitive load in VR environment is still a challenge. Traditional evaluation methods often rely on subjective questionnaires and lack objective evaluation methods. Summary of the invention

[0004] The present disclosure provides a cognitive load assessment method, apparatus, electronic device, storage medium and program product, which are used to solve at least one of the above problems.

[0005] According to a first aspect of an embodiment of the present disclosure, a cognitive load assessment method is provided, the cognitive load assessment method comprising: obtaining learning status data and answering status data of a target user, wherein the learning status data is used to describe the state changes of the target user in the process of learning a designated course, the learning status data comprises learning behavior data and physiological signal data, the learning behavior data comprises eye tracking data and facial data, the physiological signal data comprises at least one of the following: brain wave data, heart rate data, skin electricity data, blood oxygen data, the answering status data comprises designated questions answered by the target user and information on the correctness of the answers; obtaining difficulty information, wherein the difficulty information is used to indicate the difficulty of the designated course and the designated questions, the difficulty information is obtained by expert evaluation; using a concentration assessment model to process the learning status data to obtain concentration assessment information; using a knowledge tracking model to process the answering status data to obtain knowledge mastery information; using a cognitive load assessment model to process the concentration assessment information, the knowledge mastery information and the difficulty information to obtain cognitive load assessment information.

[0006] Optionally, the learning behavior data also includes action data, and the action data is used to represent the actions taken by the target user in the process of learning the designated course, and the action data includes at least one of the following: learning interruption, video learning, standing experimental operation, sitting experimental operation, and absence from the seat, wherein the learning interruption is used to indicate that the designated course is stopped midway, the video learning, the standing experimental operation and the sitting experimental operation are used to indicate the form of learning, and the absence from the seat indicates that the target user does not participate in the learning.

[0007] Optionally, the eye tracking data includes at least one of the following: eye opening and closing, pupil size, blinking frequency, and screen gaze point.

[0008] Optionally, the concentration assessment model includes an interpretable decision tree model.

[0009] Optionally, during the training phase of the concentration evaluation model, a sliding window sampling method is used to process training samples.

[0010] Optionally, the designated course includes a virtual reality course, the designated questions include questions in a question answering system, and the question answering system matches the virtual reality course.

[0011] Optionally, the designated topics include pre-learning topics and post-learning topics, the pre-learning topics are configured to be used before learning the designated course, and the post-learning topics are configured to be used after learning the designated course.

[0012] Optionally, the knowledge tracking model includes a Bayesian knowledge tracking model and a bidirectional deep knowledge tracking model, wherein using the knowledge tracking model to process the question answering data to obtain knowledge mastering information includes: using the Bayesian knowledge tracking model to process the question answering data to obtain intermediate information; using the bidirectional deep knowledge tracking model to process the intermediate information to obtain the knowledge mastering information.

[0013] According to a second aspect of an embodiment of the present disclosure, a cognitive load assessment device is provided, the cognitive load assessment device comprising: a first acquisition unit, configured to acquire learning status data and question answering status data of a target user, wherein the learning status data is used to describe the state changes of the target user in the process of learning a specified course, the learning status data comprises learning behavior data and physiological signal data, the learning behavior data comprises eye tracking data and facial data, the physiological signal data comprises at least one of the following: brain wave data, heart rate data, skin electricity data, blood oxygen data, the question answering status data comprises the designated questions answered by the target user the second acquisition unit is configured to obtain difficulty information, wherein the difficulty information is used to indicate the difficulty of the designated course and the designated question, and the difficulty information is obtained by expert evaluation; the first processing unit is configured to use the concentration evaluation model to process the learning status data to obtain concentration evaluation information; the second processing unit is configured to use the knowledge tracking model to process the question answering data to obtain knowledge mastering information; the third processing unit is configured to use the cognitive load evaluation model to process the concentration evaluation information, the knowledge mastering information and the difficulty information to obtain cognitive load evaluation information.

[0014] Optionally, the learning behavior data also includes action data, and the action data is used to represent the actions taken by the target user in the process of learning the designated course, and the action data includes at least one of the following: learning interruption, video learning, standing experimental operation, sitting experimental operation, and absence from the seat, wherein the learning interruption is used to indicate that the designated course is stopped midway, the video learning, the standing experimental operation and the sitting experimental operation are used to indicate the form of learning, and the absence from the seat indicates that the target user does not participate in the learning.

[0015] Optionally, the eye tracking data includes at least one of the following: eye opening and closing, pupil size, blinking frequency, and screen gaze point.

[0016] Optionally, the concentration assessment model includes an interpretable decision tree model.

[0017] Optionally, during the training phase of the concentration evaluation model, a sliding window sampling method is used to process training samples.

[0018] Optionally, the designated course includes a virtual reality course, the designated questions include questions in a question answering system, and the question answering system matches the virtual reality course.

[0019] Optionally, the designated topics include pre-learning topics and post-learning topics, the pre-learning topics are configured to be used before learning the designated course, and the post-learning topics are configured to be used after learning the designated course.

[0020] Optionally, the knowledge tracking model includes a Bayesian knowledge tracking model and a bidirectional deep knowledge tracking model, and the second processing unit is further configured to: use the Bayesian knowledge tracking model to process the question answering data to obtain intermediate information; use the bidirectional deep knowledge tracking model to process the intermediate information to obtain the knowledge mastery information.

[0021] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, prompt the at least one processor to execute a cognitive load assessment method according to an exemplary embodiment of the present disclosure.

[0022] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute a cognitive load assessment method according to an exemplary embodiment of the present disclosure.

[0023] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by at least one processor, prompt the at least one processor to execute a cognitive load assessment method according to an exemplary embodiment of the present disclosure.

[0024] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0025] According to the cognitive load evaluation method, device, electronic device, storage medium and program product disclosed in the present invention, based on the fact that cognitive load itself is a comprehensive concept affected by many factors and difficult to evaluate simply, the evaluation of cognitive load is broken down into three single, measurable parts, including task difficulty, the degree of active call to brain power, and task completion, and objective evaluation is performed separately, and then the three are combined to achieve an objective evaluation of the complex object of cognitive load. Specifically, by obtaining the learning status data and answering data of the target user, and using the concentration evaluation model and the knowledge tracking model for processing respectively, the concentration evaluation information and knowledge mastery information are obtained, which can objectively measure the concentration of the target user in the learning process and the final learning effect. Among them, concentration reflects the degree of active call to the target user's own brain power. The higher the concentration, the more actively the target user calls his own brain power; the knowledge mastery information reflects the task completion. The higher the degree of knowledge mastery, the higher the completion of the learning task by the target user. On this basis, combined with the difficulty information of experts' unified evaluation as a reference, the cognitive load assessment model can be used to mine the patterns in the data and analyze the extent to which the target users have completed the learning tasks through their active efforts when faced with learning tasks of a specific difficulty. This can objectively and effectively assess the cognitive load of the target users in the learning process, thereby improving the scientificity, professionalism and objectivity of cognitive load assessment.

[0026] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0028] Figure 1 is a flow chart of a cognitive load assessment method according to an exemplary embodiment of the present disclosure;

[0029] Figure 2 is a flowchart of a cognitive load assessment method according to an exemplary embodiment of the present disclosure;

[0030] Figure 3 is a block diagram of a cognitive load evaluation apparatus according to an exemplary embodiment of the present disclosure;

[0031] Figure 4 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0034] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0035] Hereinafter, a cognitive load assessment method, apparatus, electronic device, storage medium, and program product according to exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0036] Figure 1 is a flowchart of a cognitive load evaluation method according to an exemplary embodiment of the present disclosure. Figure 2 1 is a flow chart of a cognitive load evaluation method according to an exemplary embodiment of the present disclosure. The cognitive load evaluation method according to an exemplary embodiment of the present disclosure can be implemented in an electronic device with sufficient computing power.

[0037] Reference Figure 1 In step S101, the learning status data and the answering status data of the target user are obtained. It should be understood that the target user is specifically a learner user.

[0038] Specifically, the learning status data is used to describe the state changes of the target user in the process of learning a specified course, and is multimodal data. The learning status data includes learning behavior data and physiological signal data. The learning behavior data includes eye tracking data and facial data, which describes the target user's visible behavior to the naked eye. The physiological signal data includes at least one of the following: brain wave data, heart rate data, skin electricity data, and blood oxygen data, which can capture physiological signals of the target user that are invisible to the naked eye during the learning process. Combining these two aspects of information, the user's concentration in the learning process can be objectively reflected, so that the subsequent step S103 can obtain concentration evaluation information. As an example, for multimodal learning status data, timestamps can be added to all data to facilitate time alignment of the data, so as to facilitate concentration evaluation using data of different modalities collected at the same time.

[0039] Furthermore, the eye tracking data includes at least one of the following: eye opening and closing, pupil size, blinking frequency, screen fixation point, wherein the screen fixation point may further include fixation point coordinates, the distance between the fixation point and the center point of the screen, region of interest ROI (Region Of Interest), and fixation point trajectory. Regarding the fixation point and the center point of the screen, the general situation when using VR equipment is that the fixation point is the center point of the screen. During the user's scanning process, there will be a squint deviation, but the user will usually turn his head to put the target of attention in the center of the field of vision. Therefore, after the VR device is adjusted according to the head turning angle, the fixation point will return to the center point of the screen. Among these data, eye opening and closing, pupil size, coordinates of the screen fixation point, the distance between the fixation point and the center point of the screen, and region of interest ROI can be attributed to static information related to the eyes, and blinking frequency and fixation point trajectory can be attributed to dynamic information related to the eyes.

[0040] Facial data is data obtained by capturing the face of the target user, and may include basic facial video data, or facial expression data obtained by recognizing facial expressions, such as anger, confusion, disgust, fear, neutrality, happiness, sadness, surprise, etc. It may specifically include the recognized facial expression types, and may further include the possibility of recognizing that the facial expression of the target user is various expressions.

[0041] By combining eye tracking data with facial data, the focus of the target user can be comprehensively evaluated. For example, if the target user's eyes are wide open and his brows are slightly frowned, it means that the target user is very focused; and so on.

[0042] As an example, eye tracking data and facial data can be collected through extended reality devices, such as VR devices with eye tracking and facial tracking functions. If the designated course is a VR course, the VR device that comes with the VR course can be used directly without adding additional equipment. Physiological signal data can be collected through physiological signal sensors, which can be sensors equipped in multi-electrode devices, wearable devices, and portable devices.

[0043] The answering situation data includes the designated questions answered by the target user and the answering correctness information. The designated questions are related to the knowledge points learned. The answering correctness information of the designated questions can reflect the mastery of the knowledge points learned by the target user, so as to obtain the knowledge mastering information in the subsequent step S104. It should be understood that the designated course can be a specific course in a course system, and the course system may include courses of different subjects, and each course of each subject may include courses of multiple knowledge points. The designated course can be a course of one of the knowledge points, and each course of the knowledge point can have a matching topic, and the topic matching the designated course is used as the designated topic. As an example, the answering situation data includes the question number, the knowledge point number, the user number, whether the question is answered, the correctness of the answer, and the answer score. As an example, the learning state data can also be further used to describe the state changes of the target user in the process of answering the designated question. At this time, the learning state data can still add a timestamp, and the answering situation data can also add a timestamp, so that the specific question answered and its answer situation are aligned with the state of the target user in the process of answering the question, so that the concentration when answering the specific question can be aligned with the mastery of the corresponding knowledge point, which helps to improve the accuracy of cognitive load assessment.

[0044] In step S102, difficulty information is obtained.

[0045] Difficulty information is used to indicate the difficulty of a designated course and a designated question. The difficulty of a designated course specifically refers to the difficulty of understanding the knowledge points corresponding to the designated course, which may include the difficulty of the knowledge points themselves, as well as whether the explanation of the knowledge points in the designated course is clear and understandable or the clarity of the explanation. It should be understood that, as mentioned above, a designated course can be a single course corresponding to a specific knowledge point in the course system, and different courses can have corresponding difficulty information. The difficulty information is obtained by evaluating experts in the field to which the designated course belongs, and can clarify the objective difficulty of the learning task as a reference for evaluating cognitive load.

[0046] In step S103, the concentration evaluation model is used to process the learning status data to obtain concentration evaluation information.

[0047] Optionally, the concentration evaluation information can be represented by a continuous quantity, such as a score; or a discrete quantity, such as four categories of full concentration, general concentration, general inattention, and full inattention, or two categories of concentration and inattention. The present disclosure is not limited to this.

[0048] In step S104, the knowledge tracking model is used to process the answering data to obtain knowledge mastering information.

[0049] Optionally, the knowledge mastery information can be represented by a continuous quantity, such as a score, or a discrete quantity, such as fully mastered, basically mastered, not fully mastered, and not mastered, or mastered and not mastered. The present disclosure does not limit this.

[0050] In step S105, the cognitive load assessment model is used to process the concentration assessment information, the knowledge mastery information and the difficulty information to obtain the cognitive load assessment information.

[0051] As an example, the cognitive load assessment model can calculate the target user's cognitive load value under a specific learning task through a preset calculation formula or machine learning algorithm; the cognitive load value can reflect the cognitive resources required by the target user in completing the learning task, thereby assessing their learning pressure and burden. The calculated cognitive load value can also be compared with the preset threshold to determine the target user's cognitive load level, and a feedback report can be generated based on the assessment results to provide personalized learning suggestions to help the target user adjust their learning strategy, reduce cognitive burden, and improve learning results.

[0052] According to the cognitive load assessment method disclosed in the present invention, based on the fact that cognitive load itself is a comprehensive concept affected by many factors and difficult to simply assess, the assessment of cognitive load is broken down into three single, measurable parts, including task difficulty, the degree of active use of brain power, and task completion, and objectively assessed respectively. Then, the three are combined to achieve an objective assessment of the complex object of cognitive load. Specifically, referring to Figure 2, using three models to achieve two-stage data processing. In the first stage, by obtaining the target user's learning status data and answering data, and using the concentration evaluation model and knowledge tracking model to process them respectively, the concentration evaluation information and knowledge mastery information are obtained, which can objectively measure the target user's concentration in the learning process and the final learning effect. Among them, concentration reflects the degree of active use of the target user's own brain power. The higher the concentration, the more active the target user is in using his own brain power; knowledge mastery information reflects the degree of task completion. The higher the degree of knowledge mastery, the higher the degree of completion of the target user's learning task. On this basis, in the second stage, combined with the difficulty information of the expert unified evaluation as a reference, the cognitive load evaluation model can be used to mine the rules in the data, and analyze the degree of effort the target user actively puts in when facing a learning task of a specific difficulty, and to what extent the learning task is completed, so as to objectively and effectively evaluate the cognitive load of the target user in the learning process, and improve the scientificity, professionalism and objectivity of cognitive load evaluation.

[0053] As an example, when the target user is learning a simple knowledge point (indicated by difficulty information), he performs well in answering questions, and during the learning process of the corresponding knowledge point, his facial expression is relaxed and his physiological breathing is slow, which means that when learning the knowledge point, the target user has mastered the knowledge point well with low concentration, which means that he has a small cognitive load at this time, and the difficulty of the knowledge point can be adjusted appropriately and the next knowledge point can be advanced in the learning process. When the target user is learning a difficult knowledge point, he performs poorly in answering questions, and during the learning process of the corresponding knowledge point, the target user maintains a high degree of concentration, which means that the target user has a large cognitive load and needs to appropriately reduce the difficulty of the knowledge point to improve the learning effect. This can more comprehensively and intuitively reflect the learning status and cognitive load of the target user in the learning process from multiple angles, which helps to optimize the learning content arrangement of the target user, narrow the gap between the target user's learning status and the goals they want to achieve in learning and achievement, and improve the learning effect.

[0054] It should be understood that the aforementioned learning status data can be further used to describe the state changes of the target user in the process of answering designated questions, and to add timestamps to the learning status data and answer status data. Since the target user can obtain corresponding concentration assessment information, knowledge mastery information and difficulty information for each question answered during the process of answering questions, the target user's current cognitive load can be evaluated in real time during the process of answering questions, and then the overall cognitive load curve of the target user when learning specific knowledge can be drawn, thereby achieving a more refined cognitive load assessment.

[0055] In general, when training the three models used in the present disclosure, the concentration assessment model and the knowledge tracking model used in the first stage can be trained separately, and then the trained concentration assessment model and knowledge tracking model can be used to train the cognitive load assessment model used in the second stage. The training of the concentration assessment model and the knowledge tracking model will be specifically introduced in the corresponding part below. For the training of the cognitive load assessment model, as an example, a machine learning method can be used, using training samples containing sample learning status data, sample answering data and sample difficulty information, and manually marking cognitive load assessment labels in the training samples, using the trained concentration assessment model and the knowledge tracking model to process the sample learning status data and the sample answering data respectively, and obtain the sample concentration assessment information and the sample knowledge mastering information, and then combine the sample difficulty information and the cognitive load assessment label to train the cognitive load assessment model, so that the trained cognitive load assessment model can learn the data regularity between the sample concentration assessment information, the sample knowledge mastering information, the sample difficulty information and the cognitive load assessment label, and the trained cognitive load assessment model can be used to evaluate the cognitive load.

[0056] Next, we will further introduce the data processing of each of the three models.

[0057] Regarding the concentration evaluation model, optionally, the concentration evaluation model includes an interpretable decision tree model, which can effectively collect and process the multimodal learning status data of the target user during the learning process, and based on the interpretable decision tree model, accurately evaluate the concentration state of the target user when learning different knowledge points. This method not only provides real-time and objective evaluation results, but also dynamically adjusts the learning content according to the evaluation results to improve learning effects and efficiency. Of course, other models can also be used as concentration evaluation models, and this disclosure does not limit this.

[0058] Optionally, during the training phase of the focus assessment model, a sliding window sampling method is used to process the training samples. By using the sliding window sampling method, a fixed-size window can be applied to different parts of the time series to obtain data subsets in different time periods, which helps to better understand and evaluate the behavior and performance of the model.

[0059] As an example, when training a concentration assessment model, first, collect multimodal learning status data of multiple groups of learners (as sample users) during the learning process. Please refer to the above for details and will not repeat them here. In particular, according to the preset sampling frequency, continuous data samples are obtained through sliding window sampling technology. The collected raw data is filtered to remove noise and interference signals. Different types of data are standardized to ensure data consistency and comparability. The processed data is stored as a training sample for subsequent model training.

[0060] Then, based on the training samples, the interpretable decision tree model is trained and tested. In particular, the information gain, Gini index and other methods are used to select the best split features, and the interpretability score is set in combination with expert knowledge. When building a decision tree, hyperparameters such as the maximum depth and the minimum number of sample splits are set to prevent overfitting. In particular, the decision tree model is trained using the features and labels in the training samples (for example, the multimodal learning state data collected by the device when the learner learns each knowledge point), and the ensemble learning method is used to combine the results of multiple weak classifiers to improve the overall performance of the model.

[0061] Finally, based on the interpretable decision tree model, we obtain the target user's concentration evaluation information when learning different knowledge points. That is, we use the trained decision tree model to predict the target user's multimodal learning status data and obtain the target user's concentration status when learning different knowledge points. In particular, since the decision tree model has good interpretability, we can intuitively see the decision path and key features of each prediction result, which helps to provide personalized learning suggestions and feedback, adjust the learning path and content according to the evaluation results, and improve the learning effect.

[0062] In addition, from the perspective of the data basis for concentration assessment, optionally, the learning behavior data also includes action data, which is used to represent the actions taken by the target user in the process of learning a specified course, and can provide more reference for concentration assessment and cognitive load assessment. The action data includes at least one of the following: learning interruption, video learning, standing experimental operation, sitting experimental operation, and absence from the seat. As an example, the action data can be collected by a high-definition camera.

[0063] Specifically, learning interruption is used to indicate that the specified course is stopped midway, that is, the learning is not completed completely. At this time, if the knowledge mastery information obtained in step S104 indicates that the target user has a low level of knowledge mastery, it can be considered that it is related to incomplete learning, which can reduce the risk of high cognitive load assessment and improve the accuracy of the assessment. If the knowledge mastery information obtained in step S104 indicates that the target user has a high level of knowledge mastery, it can be considered that the target user may have interrupted learning because he has mastered the knowledge, so that the assessment result of cognitive load can be lowered accordingly, which can also improve the accuracy of the assessment.

[0064] Video learning, standing experimental operations, and sitting experimental operations are used to represent the forms of learning. These different learning forms may cause systematic differences in users' other learning behavior data and physiological signal data. By taking the learning form into consideration when evaluating concentration, the evaluation of concentration can be more accurate and reliable.

[0065] Not present means that the target user is not participating in the learning. In this case, the concentration and cognitive load can be omitted to reduce the risk of misassessment.

[0066] Regarding the knowledge tracking model, optionally, the knowledge tracking model includes a Bayesian knowledge tracking model and a bidirectional deep knowledge tracking model. Accordingly, step S104 includes: using the Bayesian knowledge tracking model to process the answering data to obtain intermediate information; using the bidirectional deep knowledge tracking model to process the intermediate information to obtain knowledge mastering information. By combining the Bayesian knowledge tracking model and the bidirectional deep knowledge tracking model, the answering data of the target user can be effectively collected and processed, and based on the Bayesian knowledge tracking model and the bidirectional deep knowledge tracking model, the mastery of each knowledge point of the target user when learning a specified course can be accurately evaluated. Such a method can not only provide learning feedback in real time, but also dynamically adjust the learning content to improve learning effects.

[0067] As an example, when training a knowledge tracking model, first, collect information from multiple groups of learners (as sample users) during the question-answering process, and record the learners' reaction time and answer accuracy when answering each question. Specifically, use the time window method to analyze the learners' answering situation over a period of time as a data block. Filter the collected raw data to remove noise. Standardize different types of data to ensure data consistency. Store the processed data as training samples for subsequent model training.

[0068] Then, the Bayesian knowledge tracking model is trained based on the training samples. Specifically, the answering situation in the training samples is used to initialize the parameters of the Bayesian knowledge tracking model, including the transition probability and the guessing probability. The model parameters are optimized by the maximum expectation algorithm so that the trained model can more accurately track the classroom learning situation of the target user through the answering situation. Specifically, the cross-validation method is used to evaluate the predictive performance of the model to prevent overfitting. The hidden state of the Bayesian model is updated using the learner's answering data to obtain the probability of the learner's mastery of the overall knowledge. During the model training process, the model parameters are dynamically adjusted to adapt to the learning progress and knowledge level of different learners.

[0069] Finally, based on the overall classroom learning situation of learners, the bidirectional deep knowledge tracking model is trained. Specifically, a bidirectional deep knowledge tracking model is constructed, whose input is the learner's answer sequence and output is the mastery probability of each knowledge point. The model includes two long short-term memory network layers, forward and backward, to capture the dynamic changes of learners throughout the learning process. Specifically, the sequence labeling method is used to align the learner's answer situation at each time step with the corresponding knowledge mastery status; the output of the Bayesian knowledge tracking model is used as the initial input of the Bayesian knowledge tracking model to train the bidirectional deep knowledge tracking model.

[0070] Optionally, the designated courses include virtual reality (VR) courses, which can use VR technology to provide teaching resources with richer content and easier for users to access, and can directly use a large amount of multimodal data generated in VR courses as learning status data, and combine these data with the answering data of student users in the original real education scene and the difficulty information reflecting the difficulty of knowledge, so as to provide a solid data foundation for the evaluation of cognitive load. The designated questions include questions in the answering system, and the answering system is matched with the virtual reality course to achieve the correspondence between learning content and answering questions. As an example, VR courses include but are not limited to elementary school, junior high school, high school and university courses with a large number of users, significant demand for immersive learning, and suitable for being made into VR course resources.

[0071] Optionally, the designated questions include pre-learning questions and post-learning questions, the pre-learning questions are configured to be used before learning the designated course, and the post-learning questions are configured to be used after learning the designated course. By configuring the pre-learning questions and the post-learning questions at the same time, it is necessary to ask the target user to answer the questions before and after learning the designated course, that is, to collect the answering data twice, so that by comparing the difference in the answering situation before and after learning the designated course, the changes in the target user's knowledge mastery after learning the designated course can be understood, which helps to more accurately evaluate the impact of learning the designated course on the target user's knowledge mastery, and then more accurately evaluate the target user's cognitive load in the process of learning the designated course.

[0072] In general, according to the cognitive load assessment method of the exemplary embodiment of the present disclosure, through a variety of models combined with knowledge tracking technology, combined with supporting devices and data acquisition equipment, real-time monitoring and evaluation of learners' cognitive load in VR environment, improve learners' listening effect and learning efficiency. The various models include a knowledge tracking model based on Bayesian and deep knowledge tracking, a concentration assessment model based on decision trees and interpretability weights, and a cognitive load assessment model based on expert knowledge. By collecting multimodal learning state data and answering data during the learner's learning process, a concentration assessment model based on decision trees and interpretability weights is used to process multimodal learning state data, evaluate the learner's concentration when learning different knowledge points, and use a knowledge tracking model based on Bayesian and deep knowledge tracking to process answering data to identify learners' mastery of knowledge points contained in a specific course. By adopting a cognitive load assessment model based on expert knowledge, combining the outputs of the above two models, and combining the evaluation information of the difficulty of knowledge points and corresponding test questions in a specified course by experts in the field, the assessment of cognitive load is effectively achieved. Through the cognitive load assessment method of virtual reality learning process based on multi-model knowledge tracking, the data-driven and knowledge-driven models are combined to realize real-time and comprehensive analysis and evaluation of learners' cognitive load, and obtain the overall knowledge load curve of learners when learning specific knowledge, which improves the scientificity, professionalism and objectivity of cognitive load assessment, and thus supports the improvement of learning process and teaching process (especially the teaching process in VR environment).

[0073] Figure 3 is a block diagram of a cognitive load evaluation device according to an exemplary embodiment of the present disclosure. Figure 3 The device includes a first acquisition unit 301, a second acquisition unit 302, a first processing unit 303, a second processing unit 304, and a third processing unit 305.

[0074] The first acquisition unit 301 can acquire the target user's learning status data and answering status data, wherein the learning status data is used to describe the target user's status changes in the process of learning a designated course, the learning status data includes learning behavior data and physiological signal data, the learning behavior data includes eye tracking data and facial data, the physiological signal data includes at least one of the following: brain wave data, heart rate data, skin electricity data, blood oxygen data, and the answering status data includes the designated questions answered by the target user and the correctness information of the answers.

[0075] Optionally, the learning behavior data also includes action data, which is used to represent the actions taken by the target user in the process of learning the designated course. The action data includes at least one of the following: learning interruption, video learning, standing experimental operation, sitting experimental operation, and absence from the seat, wherein learning interruption is used to indicate that the designated course is stopped midway, video learning, standing experimental operation and sitting experimental operation are used to indicate the form of learning, and absence from the seat indicates that the target user does not participate in the learning.

[0076] Optionally, the eye tracking data includes at least one of the following: eye opening, pupil size, blinking frequency, and screen gaze point.

[0077] Optionally, the designated course includes a virtual reality course, the designated questions include questions in a question answering system, and the question answering system matches the virtual reality course.

[0078] Optionally, the designated topics include pre-learning topics and post-learning topics, the pre-learning topics are configured to be used before learning the designated course, and the post-learning topics are configured to be used after learning the designated course.

[0079] The second acquisition unit 302 can acquire difficulty information, wherein the difficulty information is used to indicate the difficulty of a designated course and a designated question, and the difficulty information is obtained through expert evaluation.

[0080] The first processing unit 303 may use a concentration evaluation model to process the learning status data to obtain concentration evaluation information.

[0081] Optionally, the concentration assessment model includes an interpretable decision tree model.

[0082] Optionally, during the training phase of the concentration evaluation model, a sliding window sampling method is used to process training samples.

[0083] The second processing unit 304 may use the knowledge tracking model to process the answering data to obtain knowledge mastering information.

[0084] Optionally, the knowledge tracking model includes a Bayesian knowledge tracking model and a bidirectional deep knowledge tracking model. The second processing unit 304 can also: use the Bayesian knowledge tracking model to process the answering data to obtain intermediate information; use the bidirectional deep knowledge tracking model to process the intermediate information to obtain knowledge mastery information.

[0085] The third processing unit 305 may use the cognitive load assessment model to process the concentration assessment information, the knowledge mastery information, and the difficulty information to obtain cognitive load assessment information.

[0086] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0087] Figure 4 A structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0088] Reference Figure 4 The electronic device includes: at least one memory 401 and at least one processor 402, wherein the at least one memory 401 stores computer executable instructions, and when the computer executable instructions are executed by the at least one processor 402, the at least one processor is prompted to execute the cognitive load assessment method as described in the above exemplary embodiment.

[0089] As an example, the electronic device may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the electronic device is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instruction (or instruction set) individually or in combination. The electronic device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0090] In an electronic device, the processor 402 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. As an example and not limitation, the processor 402 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0091] The processor 402 may execute instructions or codes stored in the memory 401, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0092] The memory 401 may be integrated with the processor 402, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 401 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 401 and the processor 402 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 402 can read files stored in the memory.

[0093] In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0094] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform the cognitive load assessment method as described in the above exemplary embodiment. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0095] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including computer instructions, which, when executed by at least one processor, execute the cognitive load assessment method as described in the above exemplary embodiment.

[0096] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0097] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A cognitive load assessment method, characterized in that: The cognitive load assessment method comprises: Acquire the target user's learning status data and answering status data, wherein the learning status data is used to describe the target user's state changes in the process of learning a specified course, the learning status data includes learning behavior data and physiological signal data, the learning behavior data includes eye tracking data and facial data, the physiological signal data includes at least one of the following: brain wave data, heart rate data, skin electricity data, blood oxygen data, and the answering status data includes the specified questions answered by the target user and the correctness of the answers; Obtaining difficulty information, wherein the difficulty information is used to indicate the difficulty of the designated course and the designated question, and the difficulty information is obtained through expert evaluation; Using a concentration evaluation model, processing the learning status data to obtain concentration evaluation information; Using the knowledge tracking model, processing the answering data to obtain knowledge mastering information; The concentration assessment information, the knowledge mastery information and the difficulty information are processed using a cognitive load assessment model to obtain cognitive load assessment information.

2. The cognitive load assessment method according to claim 1, characterized in that: The learning behavior data also includes action data, and the action data is used to represent the actions taken by the target user in the process of learning the specified course, and the action data includes at least one of the following: learning interruption, video learning, standing experimental operation, sitting experimental operation, and absence from the seat, wherein the learning interruption is used to represent that the specified course is stopped midway, the video learning, the standing experimental operation, and the sitting experimental operation are used to represent the form of learning, and the absence from the seat represents that the target user does not participate in the learning; and / or The eye tracking data includes at least one of the following: eye opening, pupil size, blinking frequency, and screen gaze point.

3. The cognitive load assessment method according to claim 1, wherein: The concentration evaluation model includes an interpretable decision tree model.

4. The cognitive load assessment method according to claim 1, wherein: During the training phase of the concentration evaluation model, a sliding window sampling method is used to process training samples.

5. The cognitive load assessment method according to claim 1, wherein: The designated course includes a virtual reality course, the designated questions include questions in a question answering system, and the question answering system matches the virtual reality course; and / or The designated topics include pre-learning topics and post-learning topics. The pre-learning topics are configured to be used before learning the designated course, and the post-learning topics are configured to be used after learning the designated course.

6. The cognitive load assessment method according to claim 1, wherein: The knowledge tracking model includes a Bayesian knowledge tracking model and a bidirectional deep knowledge tracking model, wherein the use of the knowledge tracking model to process the answering data to obtain knowledge mastering information includes: Using the Bayesian knowledge tracking model, processing the answering data to obtain intermediate information; The bidirectional deep knowledge tracking model is used to process the intermediate information to obtain the knowledge mastering information.

7. A cognitive load assessment device, characterized in that: The cognitive load assessment device comprises: A first acquisition unit is configured to acquire learning status data and answering status data of a target user, wherein the learning status data is used to describe the state change of the target user in the process of learning a specified course, the learning status data includes learning behavior data and physiological signal data, the learning behavior data includes eye tracking data and facial data, the physiological signal data includes at least one of the following: brain wave data, heart rate data, skin electricity data, blood oxygen data, and the answering status data includes the specified questions answered by the target user and the correctness information of the answer; A second acquisition unit is configured to acquire difficulty information, wherein the difficulty information is used to indicate the difficulty of the designated course and the designated question, and the difficulty information is obtained by expert evaluation; A first processing unit is configured to process the learning state data using a concentration evaluation model to obtain concentration evaluation information; A second processing unit is configured to use a knowledge tracking model to process the answering data to obtain knowledge mastering information; The third processing unit is configured to use a cognitive load assessment model to process the concentration assessment information, the knowledge mastery information and the difficulty information to obtain cognitive load assessment information.

8. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the cognitive load assessment method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the cognitive load assessment method according to any one of claims 1 to 6.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by at least one processor, the at least one processor is prompted to perform the cognitive load assessment method according to any one of claims 1 to 6.

Images