Live broadcast auxiliary learning accompanying method and system based on knowledge graph

Abstract

The invention relates to the technical field of information, in particular to a live broadcast auxiliary learning accompanying method and system based on a knowledge graph. The method comprises the following steps: collecting multi-modal learning behavior data of students in a live classroom; performing fusion modeling according to the historical multi-modal learning behavior data to obtain a state recognition model; recognizing the learning state of the student under the current knowledge point according to the state recognition model, and outputting an understanding degree score; when the understanding degree score is lower than a preset reference threshold value, positioning a current knowledge point in a pre-constructed knowledge graph, and backtracking a preposed key concept node according to a knowledge dependency relationship to obtain at least one knowledge path which hinders understanding of the knowledge point; according to the knowledge path, learning accompanying content is generated, and the learning accompanying content comprises a concise concept card, a dynamic demonstration video, a case contrast question or a knowledge migration practice, and is pushed to a student terminal.

Description

Technical Field

[0001] This application relates to the field of information technology, specifically to a live-streamed assisted learning method and system based on knowledge graphs. Background Technology

[0002] In the context of the rapid development of online education, live-streaming classes have become one of the mainstream teaching formats. However, traditional live-streaming teaching generally suffers from the problems of "one-way instruction and lack of personalized feedback," making it difficult for teachers to perceive the learning status of each student in real time, let alone to provide precise intervention for individual cognitive obstacles. Although in recent years, with the development of artificial intelligence and online education, existing live-streaming teaching systems have certain interactive functions and personalized content generation capabilities, the following technical problems still exist in teaching practice: 1) Lack of a real-time learning status recognition mechanism; 2) Traditional systems cannot judge the learning comprehension through student behavior data, making it difficult for teachers to intervene in learning obstacles in a timely manner, and also lacking reasoning support based on knowledge structures; 3) Existing AI-assisted learning systems are mostly based on keyword matching or question bank retrieval, unable to generate personalized learning paths based on the logical relationships between knowledge, and unable to support dynamic review and feedback mechanisms. Teaching platforms also often fail to automatically adjust the review pace according to the students' mastery level, resulting in delayed and fragmented learning feedback.

[0003] In recent years, the development of multimodal learning analytics (MLA) technology has provided new ideas for solving the above problems. By integrating multi-source data such as facial expressions, voice tone, interactive behaviors, and operation trajectories, it is possible to more accurately model students' attention, emotions, and cognitive states. At the same time, knowledge graphs, as an effective tool for structured representation of subject knowledge systems, can clearly depict the logical dependencies between knowledge points, laying the foundation for diagnosing the root causes of learning disabilities and planning personalized remedial paths.

[0004] Therefore, it is possible to study a live-streamed learning assistance system that integrates multimodal learning behavior recognition through multimodal learning analysis, combined with knowledge graph semantic reasoning and intelligent recommendation algorithms, to achieve real-time learning monitoring and targeted knowledge supplementation in the classroom, thereby improving teaching efficiency and learning experience. Summary of the Invention

[0005] This specification describes a knowledge graph-based live-stream assisted learning method and system through several embodiments.

[0006] Firstly, the embodiments of this specification provide a live-stream assisted learning method based on knowledge graphs, including the following steps:

[0007] Collect multimodal learning behavior data of students in live-streamed classes;

[0008] A state recognition model is obtained by fusing and modeling the historical multimodal learning behavior data.

[0009] Based on the state recognition model, the student's current learning status for the knowledge point is identified, and a comprehension score is output.

[0010] When the comprehension score is lower than a preset reference threshold, the current knowledge point is located in the pre-constructed knowledge graph, and the preceding key concept nodes are traced back according to the knowledge dependency relationship to obtain at least one knowledge path that hinders the understanding of the knowledge point.

[0011] Learning materials are generated based on the knowledge path. These materials include concise concept cards, dynamic demonstration videos, case study questions, or knowledge transfer exercises, and are then pushed to the student's terminal.

[0012] Secondly, embodiments of this specification provide a knowledge graph-based live-stream assisted learning system, including:

[0013] The data acquisition module collects multimodal learning behavior data from students during live-streamed classes.

[0014] The modeling module performs fusion modeling based on the historical multimodal learning behavior data to obtain a state recognition model;

[0015] The recognition module identifies the student's current learning status for a given knowledge point based on the status recognition model and outputs a comprehension score.

[0016] The backtracking module locates the current knowledge point in the pre-constructed knowledge graph when the comprehension score is lower than a preset reference threshold, and backtracks the preceding key concept nodes according to the knowledge dependency relationship to obtain at least one knowledge path that hinders the understanding of the knowledge point.

[0017] The generation module generates supplementary learning content based on the knowledge path. The supplementary learning content includes concise concept cards, dynamic demonstration videos, case comparison questions, or knowledge transfer exercises, and pushes them to the student's terminal.

[0018] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0019] In several embodiments of this specification, the provided knowledge graph-based live-stream assisted learning method and system, by integrating students' multimodal learning behavior data in live-stream classrooms, constructs a state recognition model. This model can assess students' understanding of current knowledge points in real time, helping to overcome the limitations of relying on subjective feedback or single answer data. Combined with a pre-constructed and dynamically optimized knowledge graph, when a student's comprehension score falls below a threshold, the system automatically traces back their knowledge dependency path, accurately locating the underlying weak points that lead to comprehension difficulties. This provides a more accurate ability to extract student learning problems, offering data support for subsequent teaching. Based on the identified knowledge path and intervention focus, personalized assisted learning content adapted to different knowledge attributes is intelligently generated, realizing the generation and display of personalized teaching content.

[0020] Other features and advantages of various embodiments of this specification will be further revealed in the following detailed description and accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a diagram illustrating the live-streamed learning support provided in this manual.

[0023] Figure 2 This is a flowchart illustrating the live-streamed learning assistance method provided in this manual.

[0024] Figure 3 This is a schematic diagram of the multimodal learning behavior data provided in this specification.

[0025] Figure 4 This is a schematic diagram of the fusion modeling process provided in this specification.

[0026] Figure 5 This is a schematic diagram illustrating the process of constructing a knowledge graph as provided in this specification.

[0027] Figure 6 This is a schematic diagram of the live-streaming assisted learning system provided in this manual.

[0028] Figure 7 This is a schematic diagram of the electronic device provided in this manual. Detailed Implementation

[0029] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.

[0030] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0031] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.

[0032] All data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0033] Before introducing the technical solutions described in this manual, the application scenarios and related technologies of the technical solutions will be introduced.

[0034] In live-streaming teaching environments, teachers deliver course content to a wide range of students in real time via online platforms. This format breaks down geographical barriers, allowing more students to access high-quality educational resources. However, traditional, impersonal teaching methods may face challenges when dealing with students of varying knowledge levels. For students who have already grasped the course content well, the impersonal teaching pace may seem too slow or the content too basic, potentially leading to a decline in their learning interest and motivation.

[0035] On the other hand, for students with weaker foundational knowledge and slightly weaker comprehension skills, a uniform teaching pace and difficulty level can be difficult to keep up with. Due to the immediacy and continuity of live-streamed courses, if a certain knowledge point is not understood and digested in time, subsequent learning will become even more difficult. In this situation, students are prone to feeling frustrated and helpless, which in turn affects their overall learning outcomes.

[0036] Furthermore, students have different learning styles. Some students are better suited to visual learning materials, such as charts and videos, while others prefer auditory or hands-on learning methods. If these individual differences are not fully considered during the teaching process, it is difficult to guarantee that every student can obtain the best learning experience and results. Therefore, introducing personalized teaching mechanisms in live-streaming teaching is particularly important.

[0037] This manual provides a live-streamed learning assistance system based on Knowledge Graph 21. Please refer to the appendix. Figure 1 It can collect multimodal learning behavior data of students12, identify their learning status under the current knowledge point, and provide customized tutoring content according to the specific situation of each student. This not only helps to improve students' learning interest and efficiency, but also effectively makes up for the shortcomings of traditional non-personalized teaching models, and promotes the common progress and development of all students.

[0038] For details, please refer to the appendix. Figure 2 The live-streamed learning assistance method based on knowledge graph 21 includes the following steps:

[0039] Step S1) Collect multimodal learning behavior data of students in the live classroom 11 12.

[0040] The multimodal learning behavior data 12 includes:

[0041] The system utilizes images or video streams of students' facial expressions captured by cameras, voice signals captured by microphones, classroom interaction event logs, and screen operation trajectory data.

[0042] The student facial expression images or video streams are used to analyze attention and emotional state; the voice signals are used to extract tone, speech rate, pauses and keyword response features; the classroom interaction event log includes timestamps and content of answering questions, raising hands, chatting comments, liking or asking questions; and the screen operation trajectory data includes note-taking behavior, courseware page turning frequency, highlighted areas and mouse movement paths.

[0043] Facial images or video sequences are captured in real-time using the cameras on students' terminal devices. Computer vision technology is used to analyze students' attention span (e.g., whether their gaze is focused on the presentation area, blink frequency), emotional state (e.g., confusion, excitement, fatigue), and willingness to participate (e.g., whether they frequently look down or leave the screen). For example, during a live-streamed lesson explaining "Newton's Second Law," a student's gaze was off-screen and their brow furrowed for 30 consecutive seconds, indicating a state of "distracted attention + confusion," which may suggest difficulty understanding the current content. Students' voice input during interactive sessions (e.g., answering questions, group discussions) is collected via microphone, extracting intonation variations, speech rate changes, pause frequency, and keyword responses (e.g., whether they accurately pronounce terms like "acceleration" and "resultant force"). This reflects students' cognitive load, fluency of language expression, and level of conceptual mastery.

[0044] The classroom interaction event log records all structured interactive behaviors of students on the live streaming platform, including: answering questions (correct / incorrect, time taken), raising hands requests (frequency, timing), chat comments (content keywords, sentiment), light feedback such as likes and sending flowers, and timestamps and question types for proactive questions. For example, if a student does not participate in any interaction during the explanation of "function monotonicity," neither answering nor asking questions, while other students on the same topic interact an average of more than 3 times, this student is marked as a "low-risk individual for participation."

[0045] Screen operation trajectory data is used to track students' operational behavior on the learning interface, for example including:

[0046] Note-taking frequency and duration, slide page-turning speed (too fast may indicate skipping, too slow may indicate lag), highlighted areas of key content, and mouse movement paths (whether a particular page is repeatedly revisited). For example, if a student spends up to 5 minutes on the "Trigonometric Function Graph Transformation" slide, repeatedly highlights the "phase shift" formula, and repeatedly returns to the previous page to review basic definitions, it indicates that they are trying to build connections between concepts, but may have weak prior knowledge.

[0047] Step S2) Perform fusion modeling based on the historical multimodal learning behavior data 12 to obtain the state recognition model 13.

[0048] Please see the appendix Figure 3 The method for obtaining the state recognition model 13 by fusing and modeling based on the historical multimodal learning behavior data 12 includes:

[0049] Visual attention and emotion features are extracted from the facial expression images or video streams using a convolutional neural network.

[0050] Acoustic feature extraction is performed on the speech signal to obtain Mel spectrogram, and a temporal modeling network is used to extract intonation changes and language response patterns;

[0051] The classroom interaction event logs are sequence encoded to generate structured behavior vectors;

[0052] Spatiotemporal clustering and behavioral pattern mining are performed on the screen operation trajectory data to extract learning engagement indicators.

[0053] The visual attention and emotional features, intonation changes and language response patterns, structured behavior vectors, and learning engagement indicators are all used as input modalities. These input modalities are then input into a newly created multimodal fusion module, which uses a gating attention mechanism to dynamically weight the contributions of all input modalities.

[0054] Combining the labeled knowledge point category tags and comprehension score 14, the multimodal fusion module is trained end-to-end, and the multimodal fusion module outputs fusion features that are related to the comprehension score 14;

[0055] Based on the trained multimodal fusion module and the associated relationships, a state recognition model 13 is obtained.

[0056] Fusion modeling involves effectively fusing and semantically extracting heterogeneous, high-dimensional, and time-series multi-source behavioral signals, ultimately outputting interpretable, quantifiable, and cognitively relevant fusion features.

[0057] The acquired video streams of students' faces are first preprocessed through face detection and key point localization, and then input into a convolutional neural network (CNN) or its variants (such as ResNet and EfficientNet). The model extracts two types of key features from them: attention features, such as whether the gaze is aligned with the courseware area, head posture stability, blink frequency, etc.; and emotion features, such as discrete or continuous emotion dimensions scores such as confusion, focus, boredom, excitement, etc.

[0058] The students' speech segments are converted into Mel-spectrograms as acoustic input. Then, a temporal modeling network (such as BiLSTM, Transformer, or Wav2Vec 2.0 fine-tuning model) is used to capture: intonation changes (such as rising intonation for questions and falling intonation for confirmation), speech rate and pause patterns (long pauses may reflect thought block), and the accuracy of keyword response (whether core terms such as "concentration", "pressure", and "temperature" are said).

[0059] Discrete classroom interaction events (such as answering questions, asking questions, and liking) are organized into event sequences by timestamps and transformed into fixed-dimensional structured behavior vectors through embedding and positional encoding. For example, the correctness of answers is converted into Boolean values ​​[1, 0] (1 for correct, 0 for incorrect). The number of hands raised is converted into numerically normalized values. Question keywords are represented using semantic vectors encoded with BERT. For example, a student answered questions 3 times (2 wrong, 1 correct) on the topic of "the principle of diminishing marginal utility," did not ask any questions, and only liked one question. Their behavior vector exhibits a "low interaction + high-frequency error" pattern, suggesting a preliminary assessment of weak understanding of this topic.

[0060] We mine learning engagement metrics by reading screen operation trajectories (such as mouse movement, page turning, and highlighting) and performing spatiotemporal clustering and behavioral pattern mining (such as frequent revisiting and long dwell time) to extract learning engagement metrics, including: courseware browsing depth (whether examples are skipped), note density (number of times marked per unit time), and backtracking behavior intensity (frequency of returning to the previous page).

[0061] Multimodal fusion employs a gated attention mechanism for dynamic weighting, treating the four types of features (visual, speech, behavior, and operation) as independent modal inputs to the multimodal fusion module. This module uses a gated attention mechanism (or Cross-Modal Transformer) to automatically learn the importance weights of each modality based on the current context. For example, in the lecture phase, visual and operational trajectories have higher weights; in the question-and-answer interaction phase, speech and interaction logs have increased weights.

[0062] End-to-end training and association establishment are performed, with the fusion module outputting a high-dimensional fusion feature vector. Combining manually labeled knowledge point category tags and comprehension scores (14), end-to-end training is conducted using regression or ordered classification loss functions. The training objective is to establish a monotonic or differentiable mapping relationship between the high-dimensional fusion feature vector and the comprehension scores (14). After training, the entire multimodal fusion module and its mapping head (e.g., a linear layer) from the high-dimensional fusion feature vector to the comprehension scores (14) are solidified, forming the state recognition model (13). During the inference phase, only real-time multimodal data needs to be input to output the predicted value of the comprehension score (14) for the current knowledge point.

[0063] For details, please refer to the appendix. Figure 4 The methods for obtaining the aforementioned association include:

[0064] During the training phase of the multimodal fusion module, the fusion features output by the multimodal fusion module are aligned with the corresponding comprehension score 14 to construct a supervised learning objective;

[0065] The deviation between the understanding score 14 predicted by the fused features and the labeled understanding score 14 is calculated using a regression loss function or an ordered classification loss function.

[0066] The parameters of the multimodal fusion module are optimized by backpropagation algorithm so that the fused features have a monotonic or differentiable mapping relationship with the comprehension score 14 in the embedding space.

[0067] A contrastive learning strategy is introduced to impose distance constraints on sample pairs that come from the same knowledge point but have significantly different comprehension scores.

[0068] The mapping function or weight parameters from fused features to the understanding score 14 in the trained multimodal fusion module are solidified as the association.

[0069] During the training phase, each historical sample (i.e., a student's multimodal behavioral data on a specific knowledge point) is input into the multimodal fusion module to obtain a fused feature vector z. Simultaneously, this sample corresponds to a comprehension score 14y∈[0,1], either labeled by the teacher or generated based on the student's performance. For example, when explaining "Why don't consumers buy the same product indefinitely?", student A accurately exemplifies "Eating the first chocolate is satisfying, but the third one is cloying," and correctly draws the marginal utility curve during the interaction; the teacher labels their comprehension score as 4.7 / 5. Student B confuses "total utility" with "marginal utility," believing that "an increase in marginal utility means a decrease in total utility," answers incorrectly, and appears confused; their score is 2.3 / 5. Mean squared error is used as the loss function.

[0070] By jointly optimizing the entire multimodal fusion network through backpropagation, the fusion feature z exhibits a monotonic or differentiable mapping relationship with the comprehension score 14y in the embedding space. That is, the deeper a student's understanding of the "utility function," the higher the projection value of z in a specific direction; as students progress from "memorizing the formula" to "being able to explain why MRS equals the price ratio," their predicted comprehension should show a smooth upward trend.

[0071] A contrastive learning strategy is introduced to specifically handle sample pairs with "significant differences in understanding levels under the same knowledge point": Sample pair construction: Two students who are both learning "consumer equilibrium conditions" are selected, one with a comprehension level of 4.6 (able to derive the Lagrange method to solve the problem), and the other with only 2.1 (unable to distinguish between MRS and price ratio); Distance constraint is applied: The distance between the fused features of the two students is increased in the embedding space, even if their camera feeds both show "gazing at the screen".

[0072] For example, surface behaviors may be similar (e.g., neither raised their hand, both took notes), but deep cognitive differences are huge. Contrastive learning forces the model to pay attention to the accuracy of terminology in speech (whether "marginal rate of substitution" is said), the graph drawing behavior in the operational trajectory (whether an attempt is made to draw an indifference curve), and the depth of conversational questions ("Why is the cut point optimal?" vs. "How do I memorize this formula?"), thereby improving its ability to recognize higher-order economic thinking.

[0073] After training is completed, the mapping head from the fused feature z to the understanding y (such as a fully connected layer with Sigmoid activation) and its weight parameters are permanently saved as "associations".

[0074] Step S3) Identify the student's current learning status under the knowledge point based on the state recognition model 13, and output the comprehension score 14.

[0075] The methods for identifying a student's current learning status based on state recognition model 13 and outputting a comprehension score 14 include:

[0076] Check whether the real-time collected multimodal learning behavior data 12 is complete. If there are missing data for several modalities in the multimodal learning behavior data 12, select the sample that best matches the collected learning behavior data from the preset filling library to fill it in. The preset filling library contains a large amount of historical learning behavior data.

[0077] The imputed multimodal learning behavior data 12 is input into the state recognition model 13. Using the response of the state recognition model 13, key features reflecting the student's learning state are extracted and mapped onto a predefined comprehension score 14 scale, and the student's comprehension score 14 for the current knowledge point is output.

[0078] Along with the output comprehension score of 14, a confidence interval is added to indicate the reliability of the score. The confidence interval is calculated based on the error distribution characteristics determined during the model training phase and is provided to the system along with the score for subsequent decision-making, such as determining whether to generate and push personalized learning content 23.

[0079] In a live streaming environment, due to limitations such as device permissions, network fluctuations, or students' operating habits, not all modal data can be completely collected. For example, some students may turn off their cameras (no facial video) or disable their microphones (no audio input). To address this, the system first performs an integrity check on the real-time collected multimodal learning behavior data 12. If some modalities are detected to be missing (e.g., only interaction logs and screen operations are present, but no audio or video), an intelligent imputation mechanism is activated: from a preset imputation library (containing a large number of complete multimodal behavior samples of historical students on similar knowledge points), the system retrieves historical samples that best match the currently collected modalities; based on similarity metrics (such as cosine similarity and behavior sequence alignment scores), it selects one or more highly matched samples and imputes them using the features of their missing modalities.

[0080] For example, student A is learning about the "principle of diminishing marginal utility," but due to privacy settings, the camera is turned off, leaving only the answer record (two correct answers), slow page turning, and a question in the chat: "Why does the third slice of pizza not taste good anymore?" The system finds a sample of a student in the past who also answered the question correctly, asked a question focusing on real-life examples, and turned pages slowly in the fill-in-the-blank database, and uses their typical "focused expression" and "steady tone" features to fill in the gaps, avoiding an underestimation of comprehension due to modality loss.

[0081] The imputed multimodal data (including visual, speech, interaction, and operation features) is input into the pre-trained state recognition model 13. The multimodal fusion module within this model automatically weights the contributions of each modality and outputs a fused feature vector z. Subsequently, the fused feature vector z is mapped to a predefined understanding score 14 using a pre-defined association mapping function (such as a linear regression head or an ordered classifier). To enhance the robustness of the system's decision-making, the model appends a confidence interval (e.g., [0.82, 0.95]) to the output understanding score 14, representing the reliability of the score. The reliability is dynamically calculated based on the statistical distribution of prediction errors during model training (e.g., residual standard deviation, quantile regression results), taking into account the modal completeness and feature outlier of the current input data. If all modalities are complete and the behavior is typical, the confidence interval is narrow (high reliability); if it relies on imputation or exhibits abnormal behavior (e.g., long periods without operation), the confidence interval is wide (low reliability).

[0082] Step S4) When the comprehension score 14 is lower than the preset reference threshold, locate the current knowledge point in the pre-constructed knowledge graph 21, and backtrack the preceding key concept nodes according to the knowledge dependency relationship to obtain at least one knowledge path 22 that hinders the understanding of the knowledge point.

[0083] Please refer to the appendix for details. Figure 5 The methods for constructing knowledge graphs include:

[0084] Receive curriculum standards, textbook texts, and teaching outlines; extract subject knowledge point entities; and standardize the naming of each knowledge point and associate it with a unique identifier.

[0085] Identify the semantic dependencies between knowledge points, including sequential relationships, inclusion relationships, causal relationships, analogical relationships, and application scenarios, and construct directed edges between knowledge points;

[0086] The knowledge point entities are used as nodes and the semantic dependencies are used as edges to form a directed graph, which serves as the initial knowledge graph 21.

[0087] Read students' historical answer data and wrong answer data, and dynamically supplement implicit cognitive dependency paths through graph embedding or association rule mining methods. Based on the students' mastery and transfer strength between knowledge points, quantify the edge weights in the initial knowledge graph 21 to obtain the final knowledge graph 21.

[0088] First, core concepts are automatically extracted as knowledge point entities from the Microeconomics curriculum standards, mainstream textbooks (such as Mankiw and Pindyck), and teaching syllabus, and then standardized and uniquely identified by ID. Examples are shown in Table 1.

[0089] Table 1 Knowledge Point Entity Table

[0090]

[0091] Based on educational logic and subject structure, the system identifies semantic dependencies between knowledge points and constructs directed edges.

[0092] The sequential relationships are as follows: Principle of diminishing marginal utility → Utility function → Indifference curve → Consumer equilibrium condition. This means that marginal utility must be understood before constructing a utility function; indifference curves must be understood before analyzing optimal choices. The inclusion relationship is as follows: Consumer choice theory ⊃ {Budget constraint line, Indifference curve, Equilibrium condition}. The causal relationship is as follows: Diminishing marginal utility ⇒ Indifference curves are convex to the origin. The analogy relationship is as follows: Marginal utility ↔ Marginal product (used for cross-chapter transfer). Application scenarios are as follows: Consumer equilibrium condition ⇒ Analysis of the income and substitution effects of price changes.

[0093] Further analysis of students' historical answers and incorrect answers reveals implicit dependencies through graph embedding (e.g., Node2Vec) or association rule mining (e.g., the Apriori algorithm). Implicit paths include: data shows that many students who made mistakes on "consumer equilibrium" also made errors in "budget constraint line slope calculation," even though the textbook doesn't explicitly emphasize this dependency. The system automatically adds an edge: Budget constraint line — (implicit dependency) → Consumer equilibrium condition. Then, the edge weights are quantified: based on "mastery transfer strength" (i.e., the increase in accuracy of subsequent knowledge points after mastering prerequisite knowledge), each edge is assigned a weight w∈[0,1]: if 90% of students who master "marginal utility" can correctly understand the "utility function," the edge weight is 0.9; if only 60% of students who master "indifference curves" can solve the "equilibrium problem," the edge weight is 0.6, indicating a common cognitive bottleneck.

[0094] Methods for tracing back to key preceding concept nodes based on knowledge dependencies to obtain at least one knowledge path 22 that hinders the understanding of knowledge points include:

[0095] In the knowledge graph 21, locate the node corresponding to the current knowledge point as the target node;

[0096] Starting from the target node, a search is performed in the reverse direction along the knowledge dependency edges to extract all reachable preceding knowledge point nodes, forming a candidate dependency subgraph.

[0097] Based on the students' historical comprehension scores (14 data), each node in the candidate dependency subgraph is assigned a mastery probability value, which is calculated based on the students' past answer accuracy and review frequency on the corresponding knowledge points.

[0098] Several knowledge paths 22 are extracted from the candidate dependency subgraph, and the mastery weakness of knowledge path 22 is calculated based on edge weight, node mastery probability and path length.

[0099] Based on several knowledge paths 22 whose mastery level is higher than a preset reference threshold, at least one knowledge path 22 that hinders the understanding of knowledge points is generated.

[0100] When a student is learning about the "Consumer Equilibrium Condition" (ECON-CE-005), the system outputs a comprehension score of 0.58 (below the threshold of 0.7), triggering a process to obtain at least one knowledge path 22 that hinders comprehension of the knowledge point. The node ECON-CE-005 is found in the knowledge graph 21. Traversing backwards along all incoming edges yields the set of prerequisite knowledge points: Direct prerequisites: indifference curve (ECON-IC-004), budget constraint line (ECON-BC-002); Indirect prerequisites: utility function (ECON-UF-001) → diminishing marginal utility (ECON-MU-003), forming a candidate dependency subgraph. The student's historical performance is queried: Diminishing marginal utility: 92% accuracy rate, mastery probability = 0.92; Utility function: Previously confused with the Cobb-Douglas form, accuracy rate 65%, mastery probability = 0.65; Budget constraint line: Calculated the slope incorrectly multiple times, accuracy rate 58%, mastery probability = 0.58. Indifference curves: Able to draw the graph but do not understand the meaning of convexity; probability of mastery = 0.60.

[0101] For each path from the root node to the current knowledge point, calculate the cumulative mastery weakness (e.g., using a weighted product): Path 1: Marginal utility → Utility function → Indifference curve → Equilibrium, Weakness = (1-0.92)×(1-0.65)×(1-0.60)≈0.011. Path 2: Budget constraint line → Equilibrium. Weakness = (1-0.58)=0.42. Path 3: Utility function → Budget constraint line → Equilibrium (if cross-dependency exists). This not only reveals that students "haven't learned consumer equilibrium," but also precisely pinpoints the root cause as "a weak grasp of the budget constraint line." The diagnostic results differ from the traditional, crude intervention of "doing more exercises" and "explaining it again," providing data for generating targeted supplementary learning content23 (such as dynamically demonstrating the impact of budget line shifts on optimal choices under different incomes and prices).

[0102] Step S5) Generate supplementary learning content 23 based on the knowledge path 22. The supplementary learning content 23 includes concise concept cards, dynamic demonstration videos, case comparison questions or knowledge transfer exercises, and push them to the student terminal.

[0103] The methods for generating learning companion content 23 include:

[0104] Extract one or more nodes with the lowest probability of being identified along the path and use them as the focus of intervention.

[0105] Based on the attributes of the intervention focus, learning content 23 is matched and generated from the pre-accessed teaching resource library.

[0106] Specifically, based on the attributes of the intervention focus, supplementary learning content 23 is matched and generated from a pre-accessed teaching resource library. The method for generating supplementary learning content 23 includes:

[0107] When the intervention focus is a definitional or terminological concept, the generated learning content 23 is a concise concept card, which includes core definitions, keyword explanations, and tips on common misconceptions.

[0108] When the intervention focus is process-oriented or principle-oriented knowledge, the generated learning content 23 is a dynamic demonstration video or flowchart generated by calling the pre-connected visualization engine, and the dynamic demonstration video or flowchart shows the knowledge evolution logic.

[0109] The intervention focus is on easily confused or differentiated concept pairs. The generated learning content 23 consists of constructed case comparison questions, which include positive examples, negative examples, and prompts to guide students to conduct comparative analysis.

[0110] The intervention focus is a pair of knowledge points with transferable application value, and the generated learning content 23 is a variant question or cross-context task set in combination with the current teaching scenario.

[0111] Personalized learning content highly tailored to students' current cognitive gaps is generated and pushed to students' devices in real time. One or more knowledge points with the lowest mastery probability are extracted from knowledge path 22 as intervention focuses. For example, when a student learns about the "consumer equilibrium condition," they are diagnosed with low comprehension. Retrospective analysis reveals significant weaknesses in the preceding nodes "budget constraint line" (mastery probability only 0.52) and "marginal rate of substitution (MRS)" (mastery probability 0.58). The "budget constraint line" has the lowest mastery probability and is therefore identified as the primary intervention focus.

[0112] Based on the knowledge attribute category of the intervention focus, one of four types of supplementary learning content 23 is retrieved or dynamically generated from the pre-accessed teaching resource library: For definitional / terminological concepts, concise concept cards are generated, suitable for situations where basic terms and core definitions are unclear; for process / principle-based knowledge, dynamic demonstration videos or flowcharts are generated, suitable for knowledge of abstract mechanisms, dynamic evolution, or logical deduction; for easily confused concept pairs, case comparison questions are generated, suitable for clarifying frequently confused concepts that appear in pairs; for knowledge points with transfer value, knowledge transfer exercises are generated, such as variation / cross-context tasks, suitable for core models or methods that can be generalized and applied. The generated supplementary learning content 23 (such as concept cards, video links, and interactive questions) is immediately pushed to the student's terminal (such as the sidebar of the live broadcast window, pop-ups, or after-class task panel). After the student completes the interaction, their behavioral data (such as viewing time and answer results) is fed back to the system. This data is used to update the student's mastery probability of the intervention focus, dynamically adjust subsequent teaching strategies, or optimize the edge weights in the knowledge graph 21.

[0113] On the other hand, this specification provides a live-streamed assisted learning system based on knowledge graph 21. Please refer to the appendix. Figure 6 ,include:

[0114] The data acquisition module 100 collects multimodal learning behavior data 12 of students in the live classroom 11;

[0115] The modeling module 200 performs fusion modeling based on the historical multimodal learning behavior data 12 to obtain a state recognition model 13;

[0116] The recognition module 300 identifies the student's current learning status under the knowledge point based on the status recognition model 13 and outputs a comprehension score 14.

[0117] The backtracking module 400, when the comprehension score 14 is lower than the preset reference threshold, locates the current knowledge point in the pre-constructed knowledge graph 21, and backtracks the preceding key concept nodes according to the knowledge dependency relationship to obtain at least one knowledge path 22 that hinders the understanding of the knowledge point;

[0118] The generation module 500 generates learning content 23 based on the knowledge path 22. The learning content 23 includes concise concept cards, dynamic demonstration videos, case comparison questions or knowledge transfer exercises, and pushes them to the student terminal.

[0119] Please see Figure 7 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.

[0120] like Figure 7 As shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. The communication bus 1102 can be used to connect and communicate with the various components mentioned above. The user interface 1103 may include buttons, and optionally may include standard wired or wireless interfaces. The network interface 1104 may include, but is not limited to, a Bluetooth module, an NFC module, or a Wi-Fi module. The processor 1101 may include one or more processing cores. The processor 1101 connects to various parts within the electronic device 1100 using various interfaces and lines, and performs various functions of the routing device and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1105, and by calling data stored in the memory 1105. Optionally, the processor 1101 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 1101 may integrate one or more combinations of CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content that the display screen needs to show; and the modem is used for wireless communication.

[0121] It is understandable that the aforementioned modem may not be integrated into the processor 1101, but may be implemented using a separate chip.

[0122] The memory 1105 may include RAM or ROM. Optionally, the memory 1105 may include a non-transitory computer-readable medium. The memory 1105 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1105 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1105 may also be at least one storage device located remotely from the aforementioned processor 1101. As a computer storage medium, the memory 1105 may include an operating system, a network communication module, a user interface module, and application programs. The processor 1101 may be used to call the application programs stored in the memory 1105 and execute the methods in the above-described embodiments.

[0123] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform multiple steps as described in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0124] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the multiple steps described in the above embodiments.

[0125] Where there is no conflict, the technical features in this embodiment and implementation scheme can be combined arbitrarily.

[0126] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes multiple computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating multiple available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0127] When implemented through hardware or firmware, the aforementioned method flow is programmed into the hardware circuit to obtain the corresponding hardware circuit structure and achieve the corresponding function. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit, whose logic function is determined by the user programming the device. Designers can program a digital system onto a PLD themselves, eliminating the need for chip manufacturers to design and fabricate dedicated integrated circuit chips. Furthermore, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, similar to the software compiler used in program development. The original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There is not just one HDL, but many. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of the aforementioned hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logic method flow can be easily obtained.

[0128] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. A knowledge graph-based live-stream assisted learning method, characterized in that, Including the following steps: Collect multimodal learning behavior data of students in live-streamed classes; A state recognition model is obtained by fusing and modeling the historical multimodal learning behavior data. Based on the state recognition model, the student's current learning status for the knowledge point is identified, and a comprehension score is output. When the comprehension score is lower than a preset reference threshold, the current knowledge point is located in the pre-constructed knowledge graph, and the preceding key concept nodes are traced back according to the knowledge dependency relationship to obtain at least one knowledge path that hinders the understanding of the knowledge point. Learning materials are generated based on the knowledge path. These materials include concise concept cards, dynamic demonstration videos, case study questions, or knowledge transfer exercises, and are then pushed to the student's terminal.

2. The knowledge graph-based live-stream assisted learning method according to claim 1, characterized in that, The multimodal learning behavior data includes: The system utilizes images or video streams of students' facial expressions captured by cameras, voice signals captured by microphones, classroom interaction event logs, and screen operation trajectory data. The student facial expression images or video streams are used to analyze attention and emotional state; the voice signals are used to extract tone, speech rate, pauses and keyword response features; the classroom interaction event log includes timestamps and content of answering questions, raising hands, chatting comments, liking or asking questions; and the screen operation trajectory data includes note-taking behavior, courseware page turning frequency, highlighted areas and mouse movement paths.

3. The knowledge graph-based live-stream assisted learning method according to claim 2, characterized in that, Methods for obtaining a state recognition model by fusing and modeling historical multimodal learning behavior data include: Visual attention and emotion features are extracted from the facial expression images or video streams using a convolutional neural network. Acoustic feature extraction is performed on the speech signal to obtain Mel spectrogram, and a temporal modeling network is used to extract intonation changes and language response patterns; The classroom interaction event logs are sequence encoded to generate structured behavior vectors; Spatiotemporal clustering and behavioral pattern mining are performed on the screen operation trajectory data to extract learning engagement indicators. The visual attention and emotional features, intonation changes and language response patterns, structured behavior vectors, and learning engagement indicators are all used as input modalities. These input modalities are then input into a newly created multimodal fusion module, which uses a gating attention mechanism to dynamically weight the contributions of all input modalities. By combining the labeled knowledge point category tags and comprehension scores, the multimodal fusion module is trained end-to-end, and the multimodal fusion module outputs fusion features that are correlated with the comprehension scores; Based on the trained multimodal fusion module and the associated relationships, a state recognition model is obtained.

4. The knowledge graph-based live-stream assisted learning method according to claim 3, characterized in that, The methods for obtaining the aforementioned association include: During the training phase of the multimodal fusion module, the fusion features output by the multimodal fusion module are aligned with the corresponding comprehension scores to construct a supervised learning objective. The deviation between the understanding score predicted by the fused features and the labeled understanding score is calculated using a regression loss function or an ordered classification loss function. The parameters of the multimodal fusion module are optimized by backpropagation algorithm so that the fusion features have a monotonic or differentiable mapping relationship with the comprehension score in the embedding space. A contrastive learning strategy is introduced to impose distance constraints on sample pairs that come from the same knowledge point but have significantly different comprehension scores; The mapping function or weight parameters from fused features to understanding scores in the trained multimodal fusion module are solidified as the association.

5. The knowledge graph-based live-stream assisted learning method according to claim 3, characterized in that, Methods for identifying a student's current learning status regarding a knowledge point based on a state recognition model and outputting a comprehension score include: Check whether the real-time collected multimodal learning behavior data is complete. If there are missing data for several modalities in the multimodal learning behavior data, select the sample that best matches the collected learning behavior data from the preset filling library to fill it in. The preset filling library contains a large amount of historical learning behavior data. The imputed multimodal learning behavior data is input into the state recognition model. The response of the state recognition model is used to extract key features that reflect the student's learning state and map them onto a predefined comprehension scoring scale to output the student's comprehension score for the current knowledge point.

6. The knowledge graph-based live-stream assisted learning method according to claim 1, characterized in that, Methods for constructing knowledge graphs include: Receive curriculum standards, textbook texts, and teaching outlines; extract subject knowledge point entities; and standardize the naming of each knowledge point and associate it with a unique identifier. Identify the semantic dependencies between knowledge points, including sequential relationships, inclusion relationships, causal relationships, analogical relationships, and application scenarios, and construct directed edges between knowledge points; The knowledge point entities are used as nodes and the semantic dependencies are used as edges to form a directed graph, which serves as the initial knowledge graph. By reading students' historical answer data and incorrect answer data, implicit cognitive dependency paths are dynamically supplemented through graph embedding or association rule mining methods. Based on the students' mastery and transfer strength between knowledge points, the edge weights in the initial knowledge graph are quantified to obtain the final knowledge graph.

7. The knowledge graph-based live-stream assisted learning method according to claim 1, characterized in that, Methods for tracing back to key preceding concept nodes based on knowledge dependencies to obtain at least one knowledge path that hinders the understanding of a knowledge point include: Locate the node corresponding to the current knowledge point in the knowledge graph as the target node; Starting from the target node, a search is performed in the reverse direction along the knowledge dependency edges to extract all reachable preceding knowledge point nodes, forming a candidate dependency subgraph. Based on the students' historical comprehension score data, each node in the candidate dependency subgraph is assigned a mastery probability value, which is calculated based on the students' past answer accuracy and review frequency on the corresponding knowledge points. Several knowledge paths are extracted from the candidate dependency subgraph, and the mastery weakness of the knowledge path is calculated based on the edge weight, node mastery probability and path length. Based on several knowledge paths whose mastery level is higher than a preset reference threshold, at least one knowledge path that hinders the understanding of knowledge points is generated.

8. The knowledge graph-based live-stream assisted learning method according to claim 1, characterized in that, Methods for generating learning companion content include: Extract one or more nodes with the lowest probability of being identified along the path and use them as the focus of intervention. Based on the attributes of the intervention focus, learning content is matched and generated from a pre-accessed teaching resource library.

9. The knowledge graph-based live-stream assisted learning method according to claim 8, characterized in that, Based on the attributes of the intervention focus, supplementary learning content is matched and generated from a pre-accessed teaching resource library. The method for generating supplementary learning content includes: When the intervention focus is a definitional or terminological concept, the generated learning content is a concise concept card, which includes core definitions, keyword explanations, and tips on common misconceptions. When the intervention focus is on procedural or principle-based knowledge, the generated learning content is a dynamic demonstration video or flowchart generated by calling a pre-connected visualization engine. The dynamic demonstration video or flowchart shows the knowledge evolution logic. The intervention focus is on easily confused or clarified concept pairs, and the generated learning content consists of constructed case comparison questions, which include positive examples, negative examples, and prompts to guide students to conduct comparative analysis. The intervention focus is a pair of knowledge points with transferable application value, and the generated learning content consists of variant questions or cross-contextual tasks set in conjunction with the current teaching scenario.

10. A knowledge graph-based live-streamed assisted learning system, characterized in that: include: The data acquisition module collects multimodal learning behavior data from students during live-streamed classes. The modeling module performs fusion modeling based on the historical multimodal learning behavior data to obtain a state recognition model; The recognition module identifies the student's current learning status for a given knowledge point based on the status recognition model and outputs a comprehension score. The backtracking module locates the current knowledge point in the pre-constructed knowledge graph when the comprehension score is lower than a preset reference threshold, and backtracks the preceding key concept nodes according to the knowledge dependency relationship to obtain at least one knowledge path that hinders the understanding of the knowledge point. The generation module generates supplementary learning content based on the knowledge path. The supplementary learning content includes concise concept cards, dynamic demonstration videos, case comparison questions, or knowledge transfer exercises, and pushes them to the student's terminal.

Images