A digital smart teaching auxiliary system based on a multi-modal large model

Abstract

This invention discloses a digital intelligent teaching assistance system based on a multimodal large model. A multimodal acquisition module acquires facial expressions, speech, and action information during the student's current learning process and sends this information to a state modeling module to obtain the current attention level and confusion level. A feedback correction unit obtains a feedback correction amount based on automatic scoring, student feedback scoring, and confusion level. A mastery update module updates the student's current mastery level based on learning events, feedback amount, and feedback correction amount. A difficulty adjustment module determines a difficulty bucket based on mastery level and state variables. A task generation module outputs subsequent learning tasks based on the difficulty bucket. Within this invention's framework, learning events and multimodal states are not independent but jointly enter the mastery update module. The mastery update result further drives difficulty adjustment and task generation, thus forming a mechanism-driven structure centered on the evolution of the student's knowledge state. This clearly introduces the learning state into the path of knowledge state evolution, enabling stable task level adjustments under different learning states. Simultaneously, displaying feedback variables completes the learning loop, improving the resolution of mastery estimation, stabilizing mastery adjustment, and thus enhancing teaching effectiveness.

Description

Technical Field

[0001] This invention belongs to the field of smart teaching technology, and more specifically, relates to a digital smart teaching support system based on a multimodal large model. Background Technology

[0002] Digital intelligent teaching support systems include the following two categories:

[0003] The first category comprises large-scale educational model solutions focused on teaching content generation and lesson preparation optimization. These solutions typically revolve around teacher lesson preparation data, curriculum knowledge structure, natural language interaction, and content generation strategies, forming a closed loop through teacher input, voice interaction, updates to the teaching knowledge structure, and content optimization. Their core focus is the optimization of teaching content, lesson preparation structure, and generation strategies.

[0004] The second category is online adaptive learning and intelligent tutoring solutions. These solutions typically record students' answers, learning time, learning progress, and historical dialogue information, and based on this, recommend subsequent learning content, generate practice questions, or provide question-and-answer services. Some solutions incorporate facial expression recognition, speech analysis, or classroom behavior recognition to estimate students' attention, engagement, or emotional state.

[0005] In addition, there are virtual experimental teaching platforms that can provide experimental parameter settings, experimental process simulation, experimental result chart display, and experimental report generation. These solutions primarily demonstrate the experimental process and phenomena, but the degree of linkage between experimental results and students' knowledge acquisition, task difficulty control, and intelligent question-and-answer feedback is limited.

[0006] In summary, existing digital intelligent teaching support systems already possess the following partial capabilities: updating learning records based on answer results, generating explanations or exercises based on course content, collecting facial expressions, voice, or action information during the learning process, providing virtual experiments and experimental chart displays, and performing statistical analysis and dashboard displays on learning data.

[0007] Existing digital intelligent teaching support systems have the following shortcomings:

[0008] (1) There is no explicit mathematical link between multimodal states and mastery updates.

[0009] Existing digital intelligent teaching support systems typically use facial expressions, voice, and actions for monitoring or display, but rarely incorporate variables such as attention, confusion, and speed into the mastery update model in mathematical form. Therefore, the path by which learning state affects the evolution of knowledge state is unclear.

[0010] (2) The task difficulty adjustment mechanism lacks a unified discrete execution model.

[0011] Common solutions often use empirical thresholds, chapter order, or static rules to determine the next task, lacking a discrete difficulty bucket model driven by both mastery and real-time status, making it difficult to stably adjust the task level under different learning states.

[0012] (3) Feedback variables have not explicitly entered the learning state closed loop.

[0013] While existing large-scale teaching models can generate feedback, this feedback is mostly used for text content correction or Q&A enhancement. The mechanisms for incorporating variables such as automatic scoring, feedback consistency, and confusion into the mastery update equation are insufficient, and the closed-loop update object is usually not the student's knowledge status.

[0014] (4) The basis for updating the level of mastery is singular.

[0015] Most adaptive learning systems still use the accuracy of answers or completion status as the dominant variable, making it difficult to distinguish between different learning states such as "answering correctly but taking a long time and being highly confused" and "answering incorrectly but with a nearly correct approach," resulting in limited resolution in mastery estimation.

[0016] (5) Multimodal signals exhibit jitter and lack stabilization mechanisms.

[0017] Raw facial expressions, voice, and motion signals have instantaneous fluctuation characteristics. Without the introduction of sliding window aggregation and threshold gating, the task difficulty may switch frequently between adjacent events, leading to unstable regulation. Summary of the Invention

[0018] The purpose of this invention is to overcome the shortcomings of the prior art and provide a digital intelligent teaching assistance system based on a multimodal large model. This system clarifies the path of the learning state's influence on the evolution of the knowledge state, stably executes task level adjustments under different learning states, displays feedback variables to complete the learning loop, improves the resolution of mastery estimation, and stably adjusts mastery, thereby enhancing teaching effectiveness.

[0019] To achieve the above-mentioned objectives, the present invention provides a digital intelligent teaching support system based on a multimodal large model, characterized in that it comprises:

[0020] The multimodal acquisition module is used to acquire facial expressions, speech, and action information during the current student's learning process and output it to the state modeling module;

[0021] The state modeling module is a multimodal large model used to output attention and confusion levels based on facial expressions, speech, and motion information.

[0022] The feedback correction unit is used to obtain the feedback correction amount based on the automatic score of the current student's learning, the student's feedback score, and the level of confusion.

[0023] The mastery update module integrates attention and confusion with correctness and learning speed to obtain a basic update amount. This basic update amount is then fed into the update function along with the mastery level and feedback correction amount from the previous learning session. The current mastery level and the basic update amount are allocated according to the update coefficient, and the feedback correction amount is added according to the feedback correction weight to obtain the mastery level for the current learning session.

[0024] The difficulty adjustment module determines the difficulty level based on the current level of mastery, attention, and confusion.

[0025] The task generation module searches for learning tasks with the corresponding difficulty bucket in the task cache or generates learning tasks with the corresponding difficulty bucket based on the large model and assigns them to students for execution.

[0026] After students complete their learning tasks, the mastery level and difficulty bucket are updated again, and students are given learning tasks of the corresponding difficulty bucket to complete, thus realizing digital intelligent teaching assistance for students.

[0027] The objective of this invention is achieved as follows.

[0028] This invention presents a digital intelligent teaching support system based on a multimodal large-scale model, comprising a multimodal acquisition module, a state modeling module, a feedback correction unit, a mastery update module, a difficulty adjustment module, and a task generation module. The multimodal acquisition module acquires facial expressions, speech, and actions during the student's current learning process and sends this information to the state modeling module to obtain the current attention level and confusion level. The feedback correction unit calculates a feedback correction amount based on automatic scoring, student feedback scores, and confusion level. The mastery update module updates the student's current mastery level based on learning events, feedback amount, and feedback correction amount. The difficulty adjustment module determines the difficulty bucket based on the mastery level and state variables. The task generation module outputs subsequent learning tasks based on the difficulty bucket. Within this invention, learning events and multimodal states are not independent but jointly enter the mastery update module. The mastery update result further drives difficulty adjustment and task generation, forming a mechanism-driven structure centered on the evolution of the student's knowledge state. This clearly introduces the learning state into the path of knowledge state evolution, enabling stable task level adjustments under different learning states. Simultaneously, displaying feedback variables creates a learning loop, improving the resolution of mastery estimation, stabilizing mastery adjustment, and thus enhancing teaching effectiveness. Attached Figure Description

[0029] Figure 1 This is a general architecture diagram of a specific implementation of the digital intelligent teaching assistance system based on a multimodal large model of the present invention;

[0030] Figure 2 This is a flowchart of the event-driven update mechanism;

[0031] Figure 3 This is a flowchart of a closed-loop feedback mechanism;

[0032] Figure 4 This is a diagram of the mastery update mechanism. Detailed Implementation

[0033] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0034] Figure 1 This is a general architecture diagram of a specific implementation of the digital intelligent teaching assistance system based on a multimodal large model according to the present invention.

[0035] In this embodiment, as Figure 1 As shown, the digital intelligent teaching assistance system based on a multimodal large model of the present invention includes a multimodal acquisition module 1, a state modeling module 2, a feedback correction unit 3, a mastery update module 4, a difficulty adjustment module 5, a task generation module 6, and a task cache 7.

[0036] Multimodal acquisition module 1 acquires the current The system collects students' facial expressions, voice, and actions during the learning process and outputs them to the state modeling module 2.

[0037] State modeling module 2 is a multimodal large model that outputs attention based on facial expressions, speech, and motion information. and confusion In this embodiment, attention Attention and confusion The value range of 'confusion' is all within the range of Among them, attention With confusion By sliding window The attention obtained from the multimodal samples within the module, namely facial expressions, speech, and motion information, is obtained using the state modeling module 2. Confusion level Aggregation yields:

[0038]

[0039]

[0040] in, For sliding windows The collection time in the middle, For sliding windows Number of data collection moments, attention Confusion level The collection time is respectively Attention, confusion, and sliding window obtained from state modeling module 2 For the present The time period for students to study.

[0041] Feedback correction unit 3 is used to obtain the feedback correction amount based on automatic scoring, student feedback scoring, and confusion level.

[0042] like Figure 2 As shown, the digital intelligent teaching assistance system based on a multimodal large model of this invention adopts an event-driven update mechanism. When a learning event occurs, the system first collects and aggregates the state to obtain the student's attention and confusion level for the current learning session; then it analyzes the learning event to obtain the correctness and speed variables; then it updates the mastery level based on feedback information; and finally, it adjusts the difficulty level according to the updated mastery level and the current state, generating a learning task of the corresponding level for the student to perform. The student's performance results then constitute the next round of learning events.

[0043] By adopting the aforementioned event-driven update mechanism, the core of the system no longer manifests as a simple combination of several independent functional modules, but rather as a unified computational framework centered on state evolution. This clearly describes the source of each state variable, the event triggering conditions, and the state transition path, demonstrating the mathematical completeness and feasibility of the technical solution.

[0044] This invention employs a closed-loop feedback mechanism (prediction → execution → feedback → update). To form a closed loop, as follows... Figure 3 As shown, the present invention constructs a closed-loop mechanism consisting of prediction, execution, feedback, and updating based on a multimodal large-scale digital intelligent teaching support system. The system first predicts the task difficulty based on the current mastery level and state variables; then it generates and distributes learning tasks; after students execute the tasks, the system obtains automatic scoring, feedback information, and confusion status; finally, it re-inputs the feedback variables into the mastery update model to obtain a new mastery level and enters the next round of prediction.

[0045] The key to this closed-loop mechanism is that the feedback variables are not merely used for result display, but rather enter the mastery update function, becoming the computational input for subsequent task generation. In this way, the system can perform a state correction after each task execution, thus forming a continuously evolving closed loop of student knowledge state. Specifically:

[0046] Feedback correction unit 3 obtains the feedback correction amount based on automatic scoring, student feedback scoring, and confusion level. In this embodiment, the feedback correction amount... for:

[0047]

[0048] in, , , These are the weighting coefficients. For the system to currently Automatic scoring provided by students' learning. For students to understand the current situation The feedback scores given by students for their learning.

[0049] The mastery update module 4 fuses attention and confusion levels with correctness and learning speed to obtain a base update value. This base update value, along with the mastery level from the previous learning session and the feedback correction value, is then fed into the update function. The current mastery level and the base update value are allocated according to an update coefficient, and the feedback correction value, calculated according to the feedback correction weight, is added to obtain the mastery level for the current learning session. Specifically, in this embodiment, as follows... Figure 4 As shown, attention and confusion levels are fused with correctness and learning speed to obtain the basic update quantity:

[0050]

[0051] in: Based on update volume, Let be the weight coefficient, and satisfy... , For the present The correctness of the student's learning Learning speed, with a value range of: Basic update volume The value is located at .

[0052] In this embodiment, the weighting coefficient can be:

[0053]

[0054] Then we have:

[0055]

[0056] This basic update function makes mastery determined by learning outcomes, learning speed, attention, and confusion.

[0057] Learning speed for:

[0058]

[0059] in, For the present Duration of each learning session This is the standard completion time.

[0060] In this embodiment, the update function is:

[0061]

[0062] in, current The degree of mastery of the learning. To update the coefficients, Based on the level of mastery achieved in the previous lesson, To provide feedback and adjust the weights, and to control the degree of control. The value is located at .

[0063] The mastery update process is not a simple accumulation of accuracy, but an explicit state transition process. The system maps the correctness of learning events, completion speed, and attention and confusion levels in multimodal states to a fusion function, and then combines this with feedback correction terms to complete the mastery update. Therefore, the mastery update has a clear mathematical definition and state variable dependencies.

[0064] The difficulty adjustment module 5 determines the difficulty bucket based on the current level of mastery, attention, and confusion. Specifically, in this embodiment, a discrete bucket model is used to adjust the task hierarchy to obtain the difficulty bucket. :

[0065]

[0066] in, Indicates rounding up. The possible values ​​are:

[0067]

[0068] In this embodiment, parameters .

[0069] Difficulty Bucket The degree of continuous control and the real-time status are jointly determined to form an auditable discrete execution model.

[0070] Task generation module 6 based on difficulty buckets The system retrieves the learning task with the corresponding difficulty bucket from task cache 7, or generates a learning task with the corresponding difficulty bucket based on the large model, and assigns it to the student for execution. If the learning task with the corresponding difficulty bucket is generated based on the large model, it is stored in task cache 7 after execution.

[0071] Compared with existing technologies, the core object of this invention is the evolution of students' knowledge state, and its update object is mastery level. With Difficulty Bucket Furthermore, variables such as attention, confusion, and learning speed are explicitly written into the mastery update function and difficulty adjustment function. While similar technical solutions typically use lesson preparation content, knowledge structure weights, and generation strategies as their closed-loop objects, this invention uses the student's learning state and task execution state as its closed-loop objects, resulting in clear differences in both the technical objectives and the definitions of the state variables.

[0072] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A digital intelligent teaching support system based on a multimodal large model, characterized in that, include: The multimodal acquisition module is used to acquire facial expressions, speech, and action information during the current student's learning process and output it to the state modeling module; The state modeling module is a multimodal large model used to output attention and confusion levels based on facial expressions, speech, and motion information. The feedback correction unit is used to obtain the feedback correction amount based on the automatic score of the current student's learning, the student's feedback score, and the level of confusion. The mastery update module integrates attention and confusion with correctness and learning speed to obtain a basic update amount. This basic update amount is then fed into the update function along with the mastery level and feedback correction amount from the previous learning session. The current mastery level and the basic update amount are allocated according to the update coefficient, and the feedback correction amount is added according to the feedback correction weight to obtain the mastery level for the current learning session. The difficulty adjustment module determines the difficulty level based on the current level of mastery, attention, and confusion. The task generation module searches for learning tasks with the corresponding difficulty bucket in the task cache or generates learning tasks with the corresponding difficulty bucket based on the large model and assigns them to students for execution. After students complete their learning tasks, the mastery level and difficulty bucket are updated again, and students are given learning tasks of the corresponding difficulty bucket to complete, thus realizing digital intelligent teaching assistance for students.

2. The digital intelligent teaching assistance system based on a multimodal large model according to claim 1, characterized in that, The attention and confusion levels are respectively denoted as... , , by sliding window The attention mechanism uses the state modeling module to obtain multimodal samples, including facial expressions, speech, and motion information. Confusion level Aggregation yields: ； ； in, For sliding windows The collection time in the middle, For sliding windows Number of data collection moments, attention Confusion level The collection time is respectively Attention, confusion, and sliding window obtained from the state modeling module For the present The time period for students to study. 3.The digital smart teaching auxiliary system based on a multi-modal large model according to claim 2, characterized in that, The feedback correction amount is: ； in, For feedback correction amount, , , These are the weighting coefficients. For the system to currently Automatic scoring provided by students' learning. For students to understand the current situation The feedback scores given by students for their learning.

4. The digital smart teaching auxiliary system based on a multi-modal large model according to claim 3, characterized in that, The attention and confusion levels are fused with correctness and learning speed to obtain the basic update quantity: ； in: Based on update volume, Let be the weight coefficient, and satisfy... , For the present The correctness of the student's learning Learning speed, the learning speed for: ； in, For the present Duration of each learning session This is the standard completion time. 5.The digital smart teaching auxiliary system based on a multi-modal large model according to claim 4, characterized in that, The update function is: ； wherein, Currently the degree of mastery of the previous learning, is the update coefficient, is the degree of mastery of the previous learning, is the feedback correction weight, the degree of mastery the value of which lies in .

6. The digital intelligent teaching assistance system based on a multimodal large model according to claim 5, characterized in that, Performing task level adjustment using a discrete bucketing model results in a difficulty bucket : ； wherein denotes the ceiling function, has the value: ； wherein the parameters .

Images