Online learning adjustment method and device based on emotional state, medium and equipment

Abstract

The invention discloses an online learning adjustment method and device based on an emotional state, a medium and equipment, relates to the field of wisdom education, and is applied to the fields of financial management and digital medical businesses, and the method comprises the steps: inputting the facial expression, voice information and text input information of a learner in a learning period into different emotion recognition models, obtaining a plurality of emotion classification results; performing fusion processing on the plurality of emotion classification results to obtain an emotion state of the learner; performing learning effect evaluation on the learning behavior data of the learner, and taking the emotional state and learning effect data of the learner as current state data of the learner; and matching the current state data of the learner with a plurality of preset state reference data, determining the actual learning state of the learner, and adjusting the learning content of the learner according to the actual learning state of the learner. The emotion and the learning effect are combined, the learning content suitable for the actual state of the learner is pushed, and the learning effect of the learner is improved.

Description

Technical Field

[0001] The present invention belongs to the field of smart education technology, and in particular relates to an online learning adjustment method, device, medium and equipment based on emotional state. Background Art

[0002] With the rapid development of information technology, online education has become an important part of the modern education system. More and more learners are learning through online platforms. For example, in order to facilitate students' learning, the financial management industry has launched a financial management online education platform, and the digital medical industry has also launched a digital medical online education platform.

[0003] The current mainstream online education system mainly relies on preset learning paths and fixed knowledge point push mechanisms, and adjusts content to a certain extent based on students' answer results or learning progress. Some systems try to introduce simple learning behavior analysis modules, such as evaluating knowledge mastery based on answer accuracy or completion time, and recommending subsequent learning content accordingly.

[0004] However, these online education systems generally lack mechanisms for identifying and providing feedback on students' emotional states, making it difficult to mimic the teacher's immediate perception and response to students' emotional changes in traditional classroom instruction. Consequently, existing online education systems are unable to effectively identify students' emotional states during the learning process, making it difficult to adjust learning content based on their actual state. Summary of the Invention

[0005] In view of this, the present invention provides an online learning adjustment method, device, medium and equipment based on emotional state, the main purpose of which is to solve the problem that existing online learning methods cannot adjust learning content according to the actual state of students.

[0006] According to one aspect of the present application, a method for adjusting online learning based on emotional state is provided, the method comprising:

[0007] Obtaining facial expressions, voice information, and text input information of the learner during the learning period, and inputting the facial expressions, voice information, and text input information into different preset emotion recognition models respectively to obtain emotion classification results of each emotion recognition model;

[0008] The emotion classification results of each emotion recognition model are fused to obtain the learner's emotional state;

[0009] Acquiring the learner's learning behavior data, performing a learning effect evaluation on the learning behavior data to obtain the learner's learning effect data, and using the learner's emotional state and the learning effect data as the learner's current state data;

[0010] The current state data of the learner is matched with a plurality of preset state reference data, and the actual learning state of the learner is determined according to the matching result, and the learning content of the learner is adjusted according to the actual learning state of the learner.

[0011] Optionally, the emotion classification result includes confidence levels corresponding to a plurality of emotion categories, and fusing the emotion classification results of each emotion recognition model to obtain the learner's emotional state includes:

[0012] Based on the confidence corresponding to each emotion category of each emotion recognition model, obtaining the confidence entropy corresponding to each emotion recognition model, and based on the confidence entropy corresponding to each emotion recognition model, obtaining the reliability score corresponding to each emotion recognition model;

[0013] Based on the reliability score corresponding to each emotion recognition model, obtaining a dynamic weight corresponding to each emotion recognition model;

[0014] Based on the dynamic weight corresponding to each emotion recognition model, the confidence corresponding to each emotion category of each emotion recognition model is weighted and summed to obtain the fusion confidence of each emotion category;

[0015] The emotion category corresponding to the maximum fusion confidence value is used as the emotional state of the learner.

[0016] Optionally, the acquiring the learner's learning behavior data, performing a learning effect evaluation on the learning behavior data, and obtaining the learner's learning effect data includes:

[0017] Acquire the learner's learning behavior data, wherein the learning behavior data includes answer records, knowledge point review records, wrong question redo records, time investment records, and resource usage records;

[0018] Extracting features from the learning behavior data to obtain basic features, temporal features, and correlation features;

[0019] The basic features, the time series features and the associated features are input into a preset learning effect evaluation model to obtain the learning effect data of the learner.

[0020] Optionally, matching the learner's current state data with a plurality of preset state reference data and determining the learner's actual learning state according to the matching results includes:

[0021] Obtaining weights corresponding to the emotional state and the learning effect data, and calculating a weighted Euclidean distance between the learner's current state data and each state reference data based on the emotional state and its corresponding weight, and the learning effect data and its corresponding weight;

[0022] The learning state corresponding to the state reference data with the smallest weighted Euclidean distance is used as the actual learning state of the learner.

[0023] Optionally, the actual learning state includes an active and engaged state, a calm and indifferent state, an anxious and confused state, and a bored and depressed state. Adjusting the learning content of the learner according to the learner's actual learning state includes:

[0024] When the learner's actual learning state is an active engagement state, the extended content of the current learning content is obtained from a preset knowledge resource library, and the extended content is displayed to the learner;

[0025] When the learner's actual learning state is calm and indifferent, the wrong questions corresponding to the current learning content are obtained from the learning behavior data, and interesting related content corresponding to the wrong questions is obtained from a preset knowledge resource library, and the interesting related content is displayed to the learner;

[0026] When the learner's actual learning state is an anxious and confused state, determining the learner's doubts, dividing the relevant content of the doubts into multiple first sub-learning contents, and presenting each first sub-learning content to the learner in stages;

[0027] When the actual learning state of the learner is bored and depressed, interesting learning content of the current learning content is obtained from a preset knowledge resource library, the interesting learning content is divided into multiple second sub-learning contents, and each second sub-learning content is presented to the learner in stages.

[0028] Optionally, dividing the content related to the doubtful question into a plurality of first sub-learning contents and presenting each first sub-learning content to the learner in stages includes:

[0029] Search for relevant content of the question in the preset knowledge graph, integrate the relevant content, and present the integrated answer to the learner;

[0030] During the process of displaying the integrated answer, outputting guided question information and receiving the learner's answer;

[0031] Based on the learner's response, it is determined whether the learner's doubts are resolved. If the learner's doubts are not resolved, the relevant content is reintegrated and the reintegrated answers are presented to the learner until the learner's doubts are resolved.

[0032] Optionally, after adjusting the learner's learning content according to the learner's actual learning state, the online learning adjustment method based on emotional state further includes:

[0033] When the learner's actual learning state is an active engagement state, while displaying the expanded content to the learner, motivating sentences and images are output;

[0034] When the learner's actual learning state is calm and indifferent, while showing the learner the interesting related content, encouraging sentences and images are output;

[0035] When the learner's actual learning state is anxious or confused, each first sub-learning content is presented to the learner in stages while soothing sentences and images are output;

[0036] When the actual learning state of the learner is a bored and depressed state, each second sub-learning content is presented to the learner in stages while outputting interesting and soothing sentences and images.

[0037] According to another aspect of the present application, there is provided an online learning adjustment device based on emotional state, comprising:

[0038] An emotion recognition module is used to obtain facial expressions, voice information and text input information of learners during learning, and input the facial expressions, voice information and text input information into different preset emotion recognition models respectively to obtain emotion classification results of each emotion recognition model;

[0039] The emotional state determination module is used to fuse the emotion classification results of each emotion recognition model to obtain the learner's emotional state;

[0040] a current state determination module, configured to obtain the learner's learning behavior data, perform a learning effect evaluation on the learning behavior data, obtain the learner's learning effect data, and use the learner's emotional state and learning effect data as the learner's current state data;

[0041] The content adjustment module is used to match the learner's current state data with multiple preset state reference data, determine the learner's actual learning state based on the matching results, and adjust the learner's learning content based on the learner's actual learning state.

[0042] Optionally, the emotional state determination module is further configured to:

[0043] Based on the confidence corresponding to each emotion category of each emotion recognition model, obtaining the confidence entropy corresponding to each emotion recognition model, and based on the confidence entropy corresponding to each emotion recognition model, obtaining the reliability score corresponding to each emotion recognition model;

[0044] Based on the reliability score corresponding to each emotion recognition model, obtaining a dynamic weight corresponding to each emotion recognition model;

[0045] Based on the dynamic weight corresponding to each emotion recognition model, the confidence corresponding to each emotion category of each emotion recognition model is weighted and summed to obtain the fusion confidence of each emotion category;

[0046] The emotion category corresponding to the maximum fusion confidence value is used as the emotional state of the learner.

[0047] Optionally, the current state determination module is further configured to:

[0048] Acquire the learner's learning behavior data, wherein the learning behavior data includes answer records, knowledge point review records, wrong question redo records, time investment records, and resource usage records;

[0049] Extracting features from the learning behavior data to obtain basic features, temporal features, and correlation features;

[0050] The basic features, the time series features and the associated features are input into a preset learning effect evaluation model to obtain the learning effect data of the learner.

[0051] Optionally, the current state determination module is further configured to:

[0052] Obtaining weights corresponding to the emotional state and the learning effect data, and calculating a weighted Euclidean distance between the learner's current state data and each state reference data based on the emotional state and its corresponding weight, and the learning effect data and its corresponding weight;

[0053] The learning state corresponding to the state reference data with the smallest weighted Euclidean distance is used as the actual learning state of the learner.

[0054] Optionally, the content adjustment module is further configured to:

[0055] When the learner's actual learning state is an active engagement state, the extended content of the current learning content is obtained from a preset knowledge resource library, and the extended content is displayed to the learner;

[0056] When the learner's actual learning state is calm and indifferent, the wrong questions corresponding to the current learning content are obtained from the learning behavior data, and interesting related content corresponding to the wrong questions is obtained from a preset knowledge resource library, and the interesting related content is displayed to the learner;

[0057] When the learner's actual learning state is an anxious and confused state, determining the learner's doubts, dividing the relevant content of the doubts into multiple first sub-learning contents, and presenting each first sub-learning content to the learner in stages;

[0058] When the actual learning state of the learner is bored and depressed, interesting learning content of the current learning content is obtained from a preset knowledge resource library, the interesting learning content is divided into multiple second sub-learning contents, and each second sub-learning content is presented to the learner in stages.

[0059] Optionally, the content adjustment module is further configured to:

[0060] Search for relevant content of the question in the preset knowledge graph, integrate the relevant content, and present the integrated answer to the learner;

[0061] During the process of displaying the integrated answer, outputting guided question information and receiving the learner's answer;

[0062] Based on the learner's response, it is determined whether the learner's doubts are resolved. If the learner's doubts are not resolved, the relevant content is reintegrated and the reintegrated answers are presented to the learner until the learner's doubts are resolved.

[0063] Optionally, the emotional state-based online learning adjustment device further includes:

[0064] An emotional support module, configured to output motivational sentences and images while displaying the expanded content to the learner when the learner's actual learning state is an active and engaged state;

[0065] When the learner's actual learning state is calm and indifferent, while showing the learner the interesting related content, encouraging sentences and images are output;

[0066] When the learner's actual learning state is anxious or confused, each first sub-learning content is presented to the learner in stages while soothing sentences and images are output;

[0067] When the actual learning state of the learner is a bored and depressed state, each second sub-learning content is presented to the learner in stages while outputting interesting and soothing sentences and images.

[0068] According to another aspect of the present application, a storage medium is provided, in which at least one executable instruction is stored. The executable instruction enables a processor to perform operations corresponding to the above-mentioned online learning adjustment method based on emotional state.

[0069] According to another aspect of the present application, a computer device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0070] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the online learning adjustment method based on emotional state as described above.

[0071] By means of the above technical scheme, the present application provides an online learning adjustment method, device, medium and equipment based on emotional state, which inputs the learner's facial expressions, voice information and text input information during the learning period into preset different emotion recognition models to obtain the emotion classification results of each emotion recognition model, and fuses the emotion classification results of each emotion recognition model to obtain the learner's emotional state, evaluates the learning effect of the learner's learning behavior data to obtain the learner's learning effect data, matches the learner's emotional state and learning effect data with preset state reference data, and uses the learning state corresponding to the matched state reference data as the learner's actual learning state. The learner's learning content is adjusted according to the learner's actual learning state, and the emotion and learning effect are combined to push learning content suitable for the learner's actual state, so as to mobilize the learner's learning enthusiasm, adjust the learner's learning state, improve the learner's learning effect, and enable the learner to complete the learning task as soon as possible.

[0072] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0074] Figure 1 A flowchart of an online learning adjustment method based on emotional state provided in an embodiment of the present application is shown;

[0075] Figure 2 Another flow chart of an online learning adjustment method based on emotional state provided by an embodiment of the present application is shown;

[0076] Figure 3 A block diagram of an online learning adjustment device based on emotional state provided by an embodiment of the present application is shown;

[0077] Figure 4 A structural diagram of a computer device provided in an embodiment of the present application is shown.

[0078] In the figure: 302 - emotion recognition module; 304 - emotional state determination module; 306 - current state determination module; 308 - content adjustment module; 402 - processor; 404 - communication interface; 406 - memory; 408 - communication bus; 410 - program. DETAILED DESCRIPTION

[0079] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other.

[0080] To further illustrate the technical means and effects employed by the present invention to achieve its intended objectives, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention is provided in conjunction with the accompanying drawings and preferred embodiments. In the following description, different references to "one embodiment" or "embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0081] In order to solve the problem that existing online learning methods cannot adjust the learning content according to the actual state of the students, the embodiment of the present application provides an online learning adjustment method based on the emotional state, such as Figure 1 As shown, the method includes:

[0082] 102: Obtaining facial expressions, voice information, and text input information of the learner during the learning period, inputting the facial expressions, voice information, and text input information into different preset emotion recognition models, and obtaining emotion classification results of each emotion recognition model;

[0083] 104: Fusing the emotion classification results of each emotion recognition model to obtain the learner's emotional state;

[0084] 106: Acquire the learner's learning behavior data, perform learning effect evaluation on the learning behavior data, obtain the learner's learning effect data, and use the learner's emotional state and learning effect data as the learner's current state data;

[0085] 108: Match the learner's current state data with a plurality of preset state reference data, determine the learner's actual learning state based on the matching result, and adjust the learner's learning content based on the learner's actual learning state.

[0086] Specifically, facial expression images, voice information and text input information of learners during the learning period are obtained, and the facial expression images are input into the image-type emotion recognition model to obtain the image-type emotion classification result. The voice information is input into the voice-type emotion recognition model, and voice features (such as pitch, speaking speed, pauses) are extracted through voice recognition technology to judge the learner's emotions and obtain the voice-type emotion classification result. The text input information is input into the text-type emotion recognition model to obtain the text-type emotion classification result. Since learners' emotions are usually manifested in different aspects and in different time periods, the emotions recognized by one emotion recognition model are one-sided and not accurate enough. The multiple emotion classification results recognized by multiple emotion recognition models are fused, and the fused emotional state can comprehensively represent the learner's actual situation.

[0087] Obtain learners' learning behavior data, such as answering status, answering time, completion rate, accuracy rate, note reading status, resource search status, learning time, etc., and input these learning behavior data into the preset learning evaluation model to evaluate the learners' learning effects and obtain learning effect data.

[0088] The learner's emotional state and learning effect data are used as current state data, and the current state data is matched with multiple preset state reference data to obtain the state reference data most similar to the current state data. The learning state corresponding to the most similar state reference data is used as the learner's actual learning state, such as positive state, negative state, confused state, etc. The learning content is adjusted according to the learner's actual learning state to improve the learner's enthusiasm, adjust the learner's learning state, and improve the learning effect.

[0089] In one embodiment, employees of financial management companies or digital healthcare companies need to continuously learn new skills and improve their competitiveness during their career development. When employees learn through online financial management education or digital healthcare education, the emotional state-based online learning adjustment method of this application can adjust appropriate learning content based on the employee's emotional state, allowing employees to master new skills as quickly as possible and improve their professional competitiveness.

[0090] The systems where the target tasks of this application are located include but are not limited to financial management online learning platforms, public service online learning platforms, digital medical online learning platforms, etc. The online learning adjustment methods based on emotional state include but are not limited to financial management, public services, digital medical, and insurance business application scenarios.

[0091] The online learning adjustment method based on emotional state provided by the present application, compared with the existing technology, inputs the learner's facial expressions, voice information and text input information during the learning period into different preset emotion recognition models to obtain the emotion classification results of each emotion recognition model, fuses the emotion classification results of each emotion recognition model to obtain the learner's emotional state, evaluates the learning effect of the learner's learning behavior data to obtain the learner's learning effect data, matches the learner's emotional state and learning effect data with preset state reference data, takes the learning state corresponding to the matched state reference data as the learner's actual learning state, adjusts the learner's learning content according to the learner's actual learning state, combines emotion and learning effect, and pushes learning content suitable for the learner's actual state, thereby mobilizing the learner's learning enthusiasm, adjusting the learner's learning state, improving the learner's learning effect, and enabling the learner to complete the learning task as soon as possible.

[0092] In another embodiment of the present invention, Figure 2 As shown in the figure, the emotion classification results include the confidence levels corresponding to multiple emotion categories. The emotion classification results of each emotion recognition model are fused to obtain the learner's emotional state, including:

[0093] 202: Based on the confidence corresponding to each emotion category of each emotion recognition model, obtain the confidence entropy corresponding to each emotion recognition model; based on the confidence entropy corresponding to each emotion recognition model, obtain the reliability score corresponding to each emotion recognition model;

[0094] 204: Based on the reliability score corresponding to each emotion recognition model, obtain a dynamic weight corresponding to each emotion recognition model;

[0095] 206: Based on the dynamic weight corresponding to each emotion recognition model, the confidence corresponding to each emotion category of each emotion recognition model is weighted and summed to obtain the fusion confidence of each emotion category;

[0096] 208: The emotion category corresponding to the maximum fusion confidence is used as the learner's emotional state.

[0097] Specifically, in emotion recognition scenarios, each emotion recognition model outputs a confidence score (also called a probability, such as the probability of happiness, sadness, or depression) for each emotion category based on input data. These confidence scores can be viewed as a probability distribution p. Confidence entropy is the entropy of this distribution, reflecting the emotion recognition model's degree of certainty about which emotion category the sample belongs to. Confidence entropy is calculated using the following formula: lower entropy values ​​indicate more certainty and higher reliability.

[0098]

[0099] Among them, pi is the confidence of the model for the i-th emotion category, and n is the total number of emotion categories.

[0100] The reliability score is calculated based on the confidence entropy. In essence, it converts the "uncertainty measure" into a quantitative indicator of the "reliability measure". The lower the confidence entropy (the more certain the model is about the emotion judgment), the higher the reliability score; the higher the entropy (the more uncertain the judgment), the lower the reliability score. The reliability score is calculated using the following formula:

[0101]

[0102] The higher the reliability score of a model, the greater the weight it should be given in the ensemble decision, and the lower the score of the model, the smaller the weight it should be given. Therefore, the dynamic weight corresponding to each emotion recognition model is calculated according to the reliability score. The dynamic weight is calculated using the following formula:

[0103]

[0104] Among them, w j is the dynamic weight of emotion recognition model j, r j is the reliability score of emotion recognition model j, and k is the total number of emotion recognition models.

[0105] For each emotion category, multiply the dynamic weight corresponding to each emotion recognition model by the confidence corresponding to the emotion category to obtain the recognition probability of each emotion recognition model for the emotion category. Add the recognition probabilities of all emotion recognition models for the emotion category to obtain the fusion confidence of the emotion category. Use the same method to obtain the fusion confidence of each emotion category, and take the emotion category corresponding to the maximum fusion confidence as the learner's emotional state.

[0106] The more accurate a model's judgment on a certain type of emotion is (the lower the confidence entropy and the higher the reliability), the greater its weight, and the greater the contribution of its prediction to the fusion result, forming a positive incentive of "the strong become stronger". Through the dynamic weight fusion method, the one-size-fits-all weighting defects are avoided, the "intelligence" of integrated decision-making is achieved, and the accuracy of emotional state determination is improved.

[0107] In one embodiment, the learning behavior data of the learner is obtained, and the learning effect evaluation is performed on the learning behavior data to obtain the learning effect data of the learner, including:

[0108] Obtain learners' learning behavior data, including answer records, knowledge point review records, wrong question redo records, time investment records, and resource usage records;

[0109] Extract features from learning behavior data to obtain basic features, time series features, and correlation features;

[0110] The basic features, temporal features and associated features are input into the preset learning effect evaluation model to obtain the learner's learning effect data.

[0111] Specifically, feature extraction is performed on learners' answer records, knowledge point review records, wrong question redo records, time investment records, and resource usage records to obtain basic features, temporal features, and correlation features. Basic features include answer accuracy rate, answer time, review frequency, review duration, number of wrong question redos, improvement in redo accuracy rate, average daily study time, time period distribution, resource type proportion, and high-frequency resources. Temporal features reflect the "changing trends" and "pattern regularities" of learning behavior, such as answer speed, periodicity of review frequency, stability of study time, and regularity of redo intervals. Correlation features reflect the correlation and influence between different learning behaviors, such as the correlation between wrong answers and knowledge point review, the correlation between resource use and answer effectiveness, the correlation between time and resources, and the correlation between wrong questions and time.

[0112] The basic features, time series features and correlation features are input into the preset learning effect evaluation model to obtain the learner's learning effect data, which can reflect the learner's learning efficiency, accuracy, etc.

[0113] In one embodiment, the learner's current state data is matched with a plurality of preset state reference data, and the learner's actual learning state is determined based on the matching results, including:

[0114] Obtaining the weights corresponding to the emotional state and learning effect data, and calculating the weighted Euclidean distance between the learner's current state data and each state reference data based on the emotional state and its corresponding weight, and the learning effect data and its corresponding weight;

[0115] The learning state corresponding to the state reference data with the smallest weighted Euclidean distance is taken as the learner's actual learning state.

[0116] Specifically, the emotional state and learning effect data obtained by real-time evaluation are combined to form a two-dimensional vector (emotional state, learning effect data).

[0117] Compare this vector with each state reference vector in the preset learning state library, and use the weighted Euclidean distance formula to calculate the weighted Euclidean distance between this vector and each state reference vector:

[0118]

[0119] Among them, wi is the weight, xi is the current state data, yi is the state reference data, and n is the number of vectors, where n is 2.

[0120] By assigning weights to emotional states and learning effect numbers, with emotional states having higher weights, we obtain state reference data that is most similar to the current state and determine the learner's actual learning state, so that we can adjust the learning content according to the characteristics of the emotional state.

[0121] In one embodiment, the actual learning state includes an active and engaged state, a calm and indifferent state, an anxious and confused state, and a bored and depressed state. Adjusting the learner's learning content based on the learner's actual learning state includes:

[0122] When the learner's actual learning state is actively engaged, the extended content of the current learning content is obtained from the preset knowledge resource library and displayed to the learner;

[0123] When the learner's actual learning state is calm and indifferent, the wrong questions corresponding to the current learning content are obtained from the learning behavior data, and interesting related content corresponding to the wrong questions is obtained from the preset knowledge resource library, and the interesting related content is displayed to the learner;

[0124] When the learner's actual learning state is anxious and confused, the learner's doubts are identified, and the relevant content of the doubts is divided into multiple first sub-learning contents, and each first sub-learning content is presented to the learner in stages;

[0125] When the learner's actual learning state is bored and depressed, interesting learning content of the current learning content is obtained from the preset knowledge resource library, and the interesting learning content is divided into multiple second sub-learning contents, and each second sub-learning content is presented to the learner in stages.

[0126] Specifically, when a learner's actual learning state is actively engaged, indicating that they are focused, accurate, and efficient, they need to expand their knowledge. We retrieve expansion knowledge and exercises related to the current learning content from the pre-set knowledge resource library and push these expansion knowledge and exercises directly to the learner, allowing them to deepen their understanding of the current learning content and transform short-term focus into long-term knowledge accumulation.

[0127] When learners' actual learning state is calm and indifferent, it indicates that their behavior is becoming mechanical, such as fluctuations in accuracy of ≤5%, stable time investment but declining efficiency, and high repetition of resources. Therefore, it is necessary to provide fun consolidation through incorrect questions. From the knowledge resource library, we extract historical incorrect questions related to the current knowledge point and analyze their associated characteristics, such as frequent errors and points of conceptual confusion. These points can then be presented to learners through animations, games, and other means. This creates a fun element, transforming mechanical practice into active reflection and improving learning efficiency.

[0128] When a learner's actual learning state is anxious and confused, it indicates that the learner is engaging in frustrating behaviors, such as answering more than three questions incorrectly in a row, frequently reviewing knowledge points without making any progress, or being stuck on the same content for extended periods. Therefore, the questions that are causing the learner's frustration are broken down and addressed one by one in stages. Once the question is identified, the relevant content is divided into multiple primary learning sub-contents. These primary learning sub-contents are pushed to the learner in stages, with only one primary learning content pushed at a time. Once a primary learning content is completed, the next primary learning content is unlocked, effectively addressing the anxiety and confusion in stages.

[0129] When the learner's actual learning state is boredom and depression, it means that the learner exhibits negative behaviors, such as a sharp decrease in learning time ≥50%, a sudden drop in resource usage, frequent switching of learning content, etc. Therefore, in the knowledge resource library, interesting learning content is obtained, such as dynamic comics and game tasks that integrate learning content, and the interesting learning content is divided into multiple second sub-learning contents. The second sub-learning content is pushed to the learner in stages, and only one second sub-learning content is pushed each time. When one second sub-learning content is completed, the next second sub-learning content is unlocked, turning boredom into interest and using fun to increase attention.

[0130] In one embodiment, the content related to the question is divided into multiple first sub-learning contents, and each first sub-learning content is presented to the learner in stages, including:

[0131] Search for relevant content in the preset knowledge graph, integrate the relevant content, and present the integrated answer to the learner;

[0132] In the process of displaying the integrated answer, the system outputs guided question information and receives the learner's response;

[0133] Based on the learner's response, determine whether the learner's doubts are resolved. If the learner's doubts are not resolved, reintegrate the relevant content and present the reintegrated answers to the learner until the learner's doubts are resolved.

[0134] Specifically, after determining the learner's doubts, in order to improve the learner's learning efficiency, the relevant content of the doubts is mined through natural language processing technology (such as dependency syntax analysis, entity relationship extraction, etc.) to explore the correlation between knowledge points, and then integrated to obtain integrated answers. The integrated answers are pushed to the learners. At the same time as the integrated answers are pushed, guided question information is output. According to the learner's reply, it is determined whether the learner's doubts are resolved. If the learner's doubts are not resolved, the relevant content is re-integrated until the learner's doubts are resolved, and the first sub-learning content of the next stage is output. The above process is repeated until all the first sub-learning content has been pushed.

[0135] In one embodiment, an intelligent dialogue system is built using the pre-trained dialogue model BERT, and the dialogue context is managed through state tracking technology to achieve natural and smooth multi-round dialogue. The subject knowledge graph is combined with the dialogue system to support students in asking questions and answering them.

[0136] In one embodiment, virtual human technology is used to create a virtual learning partner image, and the tone and behavior of the virtual partner are dynamically adjusted in combination with emotion recognition results.

[0137] In one embodiment, after adjusting the learner's learning content according to the learner's actual learning state, the online learning adjustment method based on emotional state further includes:

[0138] When the learner's actual learning state is actively engaged, while showing the learner expanded content, output motivating sentences and images;

[0139] When the learner's actual learning state is calm and indifferent, while showing the learner interesting and relevant content, output encouraging sentences and images;

[0140] When the learner's actual learning state is anxious and confused, each first sub-learning content is presented to the learner in stages while soothing sentences and images are output;

[0141] When the learner's actual learning state is boredom and depression, each second sub-learning content is presented to the learner in stages, while outputting interesting and soothing sentences and images.

[0142] Specifically, when the learner's actual learning state is an actively engaged state, it means that the learner is very focused on learning, has a high accuracy rate, and has high learning efficiency. Knowledge expansion is needed, and motivation is also needed. Motivational sentences and images are output to strengthen the positive state with motivation so that the learner can continue to maintain a positive attitude.

[0143] When the learner's actual learning state is calm and indifferent, it means that the learner's behavior tends to be mechanical and needs to be consolidated through interesting learning of wrong questions. At the same time, encouragement is needed, and encouraging sentences and images are output to increase the learner's confidence, break the dullness and become positive.

[0144] When the learner's actual learning state is anxious and confused, it means that the learner has frustrating behavior. The doubts that cause the learner to feel frustrated will be broken down and solved one by one in stages. The relevant content of the doubts will be divided into multiple first sub-learning contents, and the first sub-learning contents will be pushed to the learner in stages. At the same time, soothing sentences and images will be output to relieve the learner's anxiety.

[0145] When the learner's actual learning state is boredom and depression, it means that the learner is showing negative behavior. The interesting related content is divided into multiple second sub-learning contents, and the second sub-learning contents are pushed to the learner in stages. At the same time, soothing and interesting sentences and images are output to reduce the learner's psychological resistance through soothing and increase the learner's learning interest through fun.

[0146] Furthermore, as a response to the above Figure 1 The implementation of the method shown, such as Figure 3 As shown, an embodiment of the present invention provides an online learning adjustment device based on emotional state, comprising:

[0147] The emotion recognition module 302 is used to obtain the learner's facial expressions, voice information and text input information during the learning period, input the facial expressions, voice information and text input information into different preset emotion recognition models, and obtain the emotion classification results of each emotion recognition model;

[0148] The emotional state determination module 304 is used to fuse the emotion classification results of each emotion recognition model to obtain the learner's emotional state;

[0149] The current state determination module 306 is used to obtain the learner's learning behavior data, perform learning effect evaluation on the learning behavior data, obtain the learner's learning effect data, and use the learner's emotional state and learning effect data as the learner's current state data;

[0150] The content adjustment module 308 is used to match the learner's current state data with multiple preset state reference data, determine the learner's actual learning state based on the matching results, and adjust the learner's learning content based on the learner's actual learning state.

[0151] The online learning adjustment device based on emotional state provided by the present application, compared with the existing technology, inputs the learner's facial expressions, voice information and text input information during the learning period into different preset emotion recognition models to obtain the emotion classification results of each emotion recognition model, fuses the emotion classification results of each emotion recognition model to obtain the learner's emotional state, evaluates the learning effect of the learner's learning behavior data to obtain the learner's learning effect data, matches the learner's emotional state and learning effect data with preset state reference data, takes the learning state corresponding to the matched state reference data as the learner's actual learning state, adjusts the learner's learning content according to the learner's actual learning state, combines emotions and learning effects, and pushes learning content suitable for the learner's actual state, thereby mobilizing the learner's learning enthusiasm, adjusting the learner's learning state, improving the learner's learning effect, and enabling the learner to complete the learning task as soon as possible.

[0152] In one embodiment, the emotional state determination module is further configured to:

[0153] Based on the confidence corresponding to each emotion category of each emotion recognition model, the confidence entropy corresponding to each emotion recognition model is obtained, and based on the confidence entropy corresponding to each emotion recognition model, the reliability score corresponding to each emotion recognition model is obtained;

[0154] Based on the reliability score corresponding to each emotion recognition model, a dynamic weight corresponding to each emotion recognition model is obtained;

[0155] Based on the dynamic weight corresponding to each emotion recognition model, the confidence corresponding to each emotion category of each emotion recognition model is weighted and summed to obtain the fusion confidence of each emotion category;

[0156] The emotion category corresponding to the maximum fusion confidence is taken as the learner's emotional state.

[0157] In one embodiment, the current state determination module is further configured to:

[0158] Obtain learners' learning behavior data, including answer records, knowledge point review records, wrong question redo records, time investment records, and resource usage records;

[0159] Extract features from learning behavior data to obtain basic features, time series features, and correlation features;

[0160] The basic features, temporal features and associated features are input into the preset learning effect evaluation model to obtain the learner's learning effect data.

[0161] In one embodiment, the current state determination module is further configured to:

[0162] Obtaining the weights corresponding to the emotional state and learning effect data, and calculating the weighted Euclidean distance between the learner's current state data and each state reference data based on the emotional state and its corresponding weight, and the learning effect data and its corresponding weight;

[0163] The learning state corresponding to the state reference data with the smallest weighted Euclidean distance is taken as the learner's actual learning state.

[0164] In one embodiment, the content adjustment module is further configured to:

[0165] When the learner's actual learning state is actively engaged, the extended content of the current learning content is obtained from the preset knowledge resource library and displayed to the learner;

[0166] When the learner's actual learning state is calm and indifferent, the wrong questions corresponding to the current learning content are obtained from the learning behavior data, and interesting related content corresponding to the wrong questions is obtained from the preset knowledge resource library, and the interesting related content is displayed to the learner;

[0167] When the learner's actual learning state is anxious and confused, the learner's doubts are identified, and the relevant content of the doubts is divided into multiple first sub-learning contents, and each first sub-learning content is presented to the learner in stages;

[0168] When the learner's actual learning state is bored and depressed, interesting learning content of the current learning content is obtained from the preset knowledge resource library, and the interesting learning content is divided into multiple second sub-learning contents, and each second sub-learning content is presented to the learner in stages.

[0169] In one embodiment, the content adjustment module is further configured to:

[0170] Search for relevant content in the preset knowledge graph, integrate the relevant content, and present the integrated answer to the learner;

[0171] In the process of displaying the integrated answer, the system outputs guided question information and receives the learner's response;

[0172] Based on the learner's response, determine whether the learner's doubts are resolved. If the learner's doubts are not resolved, reintegrate the relevant content and present the reintegrated answers to the learner until the learner's doubts are resolved.

[0173] In one embodiment, the online learning adjustment device based on emotional state further includes:

[0174] The emotional support module is used to output motivational sentences and images while showing the learner extended content when the learner is actively engaged in learning;

[0175] When the learner's actual learning state is calm and indifferent, while showing the learner interesting and relevant content, output encouraging sentences and images;

[0176] When the learner's actual learning state is anxious and confused, each first sub-learning content is presented to the learner in stages while soothing sentences and images are output;

[0177] When the learner's actual learning state is boredom and depression, each second sub-learning content is presented to the learner in stages, while outputting interesting and soothing sentences and images.

[0178] According to one embodiment of the present invention, a storage medium is provided, which stores at least one executable instruction. The computer-executable instruction can execute the online learning adjustment method based on emotional state in any of the above method embodiments.

[0179] Figure 4 A schematic structural diagram of a computer device provided according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0180] like Figure 4 As shown, the computer device may include: a processor 402 , a communications interface 404 , a memory 406 , and a communication bus 408 .

[0181] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0182] The communication interface 404 is used to communicate with other devices such as clients or other servers.

[0183] The processor 402 is configured to execute the program 410 , and specifically to execute the relevant steps in the above-mentioned embodiment of the online learning adjustment method based on emotional state.

[0184] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0185] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in a computer device may be processors of the same data type, such as one or more CPUs, or processors of different data types, such as one or more CPUs and one or more ASICs.

[0186] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0187] The program 410 may be specifically configured to cause the processor 402 to perform the following operations:

[0188] Obtain learners' facial expressions, voice information, and text input information during learning, input the facial expressions, voice information, and text input information into different preset emotion recognition models, and obtain emotion classification results of each emotion recognition model;

[0189] The emotion classification results of each emotion recognition model are fused to obtain the learner's emotional state;

[0190] Acquire the learner's learning behavior data, conduct learning effect evaluation on the learning behavior data, obtain the learner's learning effect data, and use the learner's emotional state and learning effect data as the learner's current state data;

[0191] The learner's current state data is matched with a plurality of preset state reference data, and the learner's actual learning state is determined according to the matching results, and the learner's learning content is adjusted according to the learner's actual learning state.

[0192] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. An online learning adjustment method based on emotional state, characterized in that: include: Obtaining facial expressions, voice information, and text input information of the learner during the learning period, and inputting the facial expressions, voice information, and text input information into different preset emotion recognition models respectively to obtain emotion classification results of each emotion recognition model; The emotion classification results of each emotion recognition model are fused to obtain the learner's emotional state; Acquiring the learner's learning behavior data, performing a learning effect evaluation on the learning behavior data to obtain the learner's learning effect data, and using the learner's emotional state and the learning effect data as the learner's current state data; The current state data of the learner is matched with a plurality of preset state reference data, and the actual learning state of the learner is determined according to the matching result, and the learning content of the learner is adjusted according to the actual learning state of the learner.

2. The online learning adjustment method based on emotional state according to claim 1, characterized in that: The emotion classification result includes the confidence levels corresponding to the multiple emotion categories. The emotion classification results of each emotion recognition model are fused to obtain the learner's emotional state, including: Based on the confidence corresponding to each emotion category of each emotion recognition model, obtaining the confidence entropy corresponding to each emotion recognition model, and based on the confidence entropy corresponding to each emotion recognition model, obtaining the reliability score corresponding to each emotion recognition model; Based on the reliability score corresponding to each emotion recognition model, obtaining a dynamic weight corresponding to each emotion recognition model; Based on the dynamic weight corresponding to each emotion recognition model, the confidence corresponding to each emotion category of each emotion recognition model is weighted and summed to obtain the fusion confidence of each emotion category; The emotion category corresponding to the maximum fusion confidence value is used as the emotional state of the learner.

3. The online learning adjustment method based on emotional state according to claim 2, characterized in that: The acquiring of the learner's learning behavior data, and performing a learning effect evaluation on the learning behavior data to obtain the learner's learning effect data include: Acquire the learner's learning behavior data, wherein the learning behavior data includes answer records, knowledge point review records, wrong question redo records, time investment records, and resource usage records; Extracting features from the learning behavior data to obtain basic features, temporal features, and correlation features; The basic features, the time series features and the associated features are input into a preset learning effect evaluation model to obtain the learning effect data of the learner.

4. The online learning adjustment method based on emotional state according to claim 1, characterized in that: The matching of the learner's current state data with a plurality of preset state reference data and determining the learner's actual learning state according to the matching result includes: Obtaining weights corresponding to the emotional state and the learning effect data, and calculating a weighted Euclidean distance between the learner's current state data and each state reference data based on the emotional state and its corresponding weight, and the learning effect data and its corresponding weight; The learning state corresponding to the state reference data with the smallest weighted Euclidean distance is used as the actual learning state of the learner.

5. The online learning adjustment method based on emotional state according to claim 4, characterized in that: The actual learning state includes an active and engaged state, a calm and indifferent state, an anxious and confused state, and a bored and depressed state. Adjusting the learning content of the learner according to the learner's actual learning state includes: When the learner's actual learning state is an active engagement state, the extended content of the current learning content is obtained from a preset knowledge resource library, and the extended content is displayed to the learner; When the learner's actual learning state is calm and indifferent, the wrong questions corresponding to the current learning content are obtained from the learning behavior data, and interesting related content corresponding to the wrong questions is obtained from a preset knowledge resource library, and the interesting related content is displayed to the learner; When the learner's actual learning state is an anxious and confused state, determining the learner's doubts, dividing the relevant content of the doubts into multiple first sub-learning contents, and presenting each first sub-learning content to the learner in stages; When the actual learning state of the learner is bored and depressed, interesting learning content of the current learning content is obtained from a preset knowledge resource library, the interesting learning content is divided into multiple second sub-learning contents, and each second sub-learning content is presented to the learner in stages.

6. The online learning adjustment method based on emotional state according to claim 5, characterized in that: The step of dividing the relevant content of the doubtful question into a plurality of first sub-learning contents and presenting each first sub-learning content to the learner in stages includes: Search for relevant content of the question in the preset knowledge graph, integrate the relevant content, and present the integrated answer to the learner; During the process of displaying the integrated answer, outputting guided question information and receiving the learner's answer; Based on the learner's response, it is determined whether the learner's doubts are resolved. If the learner's doubts are not resolved, the relevant content is reintegrated and the reintegrated answers are presented to the learner until the learner's doubts are resolved.

7. The online learning adjustment method based on emotional state according to claim 5, characterized in that: After adjusting the learner's learning content according to the learner's actual learning state, the online learning adjustment method based on emotional state further includes: When the learner's actual learning state is an active engagement state, while displaying the expanded content to the learner, motivating sentences and images are output; When the learner's actual learning state is calm and indifferent, while showing the learner the interesting related content, encouraging sentences and images are output; When the learner's actual learning state is anxious or confused, each first sub-learning content is presented to the learner in stages while soothing sentences and images are output; When the actual learning state of the learner is a bored and depressed state, each second sub-learning content is presented to the learner in stages while outputting interesting and soothing sentences and images.

8. An online learning adjustment device based on emotional state, characterized in that: include: An emotion recognition module is used to obtain facial expressions, voice information and text input information of learners during learning, and input the facial expressions, voice information and text input information into different preset emotion recognition models respectively to obtain emotion classification results of each emotion recognition model; The emotional state determination module is used to fuse the emotion classification results of each emotion recognition model to obtain the learner's emotional state; a current state determination module, configured to obtain the learner's learning behavior data, perform a learning effect evaluation on the learning behavior data, obtain the learner's learning effect data, and use the learner's emotional state and learning effect data as the learner's current state data; The content adjustment module is used to match the learner's current state data with multiple preset state reference data, determine the learner's actual learning state based on the matching results, and adjust the learner's learning content based on the learner's actual learning state.

9. A storage medium storing at least one executable instruction, characterized in that: The executable instructions enable the processor to perform operations corresponding to the online learning adjustment method based on emotional state as described in any one of claims 1 to 7.

10. A computer device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, characterized in that the executable instruction enables the processor to perform operations corresponding to the online learning adjustment method based on emotional state as described in any one of claims 1-7.

Images