Classroom teaching feedback and evaluation method and system based on multi-modal large model

Abstract

The application discloses a classroom teaching feedback and evaluation method and system based on a multi-modal large model, and belongs to the technical field of education informatization and artificial intelligence auxiliary teaching evaluation. The method performs event analysis on multi-modal data in a classroom, generates evidence capsules and constructs a classroom teaching evaluation graph, retrieves an evidence set under the constraints of evidence capsule numbering and time coverage, generates controlled feedback by a multi-modal large model after scoring calibration, and performs evidence backtracking, dimension consistency and suggestion executability verification, so that the traceability, stability and reviewability of classroom evaluation are improved.

Description

Technical Field

[0001] This invention relates to the field of educational informatization and artificial intelligence-assisted teaching evaluation technology, and in particular to a method and system for classroom teaching feedback and evaluation based on a multimodal large model. Background Technology

[0002] Classroom teaching feedback and evaluation are crucial for teachers to improve classroom organization, optimize teacher-student interaction, and enhance the quality of learning feedback. Current methods largely rely on manual lesson observation and evaluation, post-lesson review, or analysis of single data indicators. These methods are time-consuming, difficult to achieve high-frequency coverage in regular classrooms, and struggle to ensure consistency in evaluation criteria among different evaluators.

[0003] With the application of audio and video capture, whiteboard recognition, courseware recording, and interactive terminal data, classroom assessment can obtain multimodal information. However, this information suffers from problems such as time asynchrony, unstable subject correspondence, cross-modal alignment errors, and numerous redundant segments. If the entire long context of the lesson is directly input into a general large model for summarization, the model is easily affected by noise and irrelevant information, and the output conclusions lack traceable evidence, resulting in insufficient interpretability and feasibility.

[0004] Therefore, there is a need for a classroom teaching feedback and evaluation method and system that can address the shortcomings of the existing technologies. Summary of the Invention

[0005] One objective of this invention is to propose a method and system for classroom teaching feedback and evaluation based on a multimodal large model. Addressing the problems of high cost, difficulty in achieving routine coverage, inconsistent evaluation criteria, lack of traceability of evidence, and unstable output from long-context large models in existing classroom teaching feedback and evaluation systems, this invention proposes a technical solution combining classroom event sequences, evidence capsules, classroom teaching evaluation maps, evidence sets, scoring calibration, and controlled feedback verification. This approach effectively improves the traceability, stability, and verifiability of evaluation conclusions.

[0006] This invention provides a classroom teaching feedback and evaluation method based on a multimodal large model, including:

[0007] S1. Acquire classroom multimodal data, perform time synchronization, privacy desensitization, classroom event parsing, and classroom stage identification on the classroom multimodal data, and generate classroom event sequences and classroom contexts derived from the classroom event sequences;

[0008] S2 converts classroom events in the classroom event sequence into evidence capsules, and constructs a classroom teaching evaluation map based on classroom events, evidence capsules, evaluation dimensions, and classroom stages. The evidence capsules include time location, modality source, subject identifier, evaluation dimension association information, confidence and alignment information, source data backlink identifier, and evidence capsule number.

[0009] S3, for the preset evaluation dimensions, the evaluation rubric is broken down into sub-index queries. Relevant event nodes, evidence capsule nodes and related edges are retrieved in the classroom teaching evaluation graph to form a candidate evidence subgraph. Under the constraint that each sub-index query corresponds to at least one evidence capsule number and meets the preset time coverage condition, the evidence set is selected.

[0010] S4: Input the evidence set into the discriminant scoring model to generate candidate evaluation results, and perform uncertainty calibration on the candidate evaluation results, outputting calibration results including scoring interval, confidence level and review risk level;

[0011] S5 inputs the evidence set, candidate evaluation results, calibration results, and classroom context into the multimodal large model. It generates explanatory feedback containing evaluation conclusions, evidence capsule numbers, causal explanations, and actionable suggestions according to the controlled feedback generation template. It performs evidence backlink verification, dimensional consistency verification, and suggestion actionability verification on the explanatory feedback. Based on the verification results, it outputs at least one of the following: real-time classroom prompts and post-class comprehensive evaluation. If the verification fails, it performs at least one of the following: regeneration, downgraded output, and manual review.

[0012] Optionally, S1 includes:

[0013] Align classroom audio, classroom video, blackboard images, courseware pages, and interactive terminal records based on the timestamps of the acquisition devices and the unified timeline of the classroom.

[0014] Perform de-identification and replacement on teacher names, student names, facial regions, student IDs, and account identifiers, while retaining the de-identification mapping version number;

[0015] Classroom events are categorized based on speaking rounds, periods of silence, changes in blackboard writing, changes in courseware pages, and records triggered by interactive terminals.

[0016] Classroom events are labeled with stage tags based on a set of stage tags for class introduction, knowledge delivery, questioning and interaction, practice feedback, and class summary;

[0017] The classroom context is generated based on the classroom event sequence, classroom stage tags, anonymized courseware page information, and interactive terminal records.

[0018] Optionally, S2 includes:

[0019] Generating the evidence capsule includes: assigning evidence capsule numbers to classroom events;

[0020] Extract speech-transcribed segments from the speech recognition results that correspond to the start and end times of the event;

[0021] Extract video keyframes from the video stream that correspond to the start and end times of the event;

[0022] Locate the area corresponding to the classroom event from the blackboard image and courseware page;

[0023] The modal confidence level is calculated based on the identifier output probability, occlusion ratio, transcription confidence value, and device synchronization deviation.

[0024] Cross-modal alignment scores are calculated based on temporal overlap ratio, semantic matching score, and subject identifier consistency.

[0025] Calculate the hash value of the source data fragment and write it into the source data back-chain identifier;

[0026] The evaluation dimension association information is generated based on the matching relationship between the evaluation dimension rubric and classroom event content, classroom stage tags, and subject identifiers;

[0027] Furthermore, constructing the classroom teaching evaluation map includes: establishing classroom event nodes, evidence capsule nodes, evaluation dimension nodes, sub-indicator nodes, classroom stage nodes, and behavioral subject nodes;

[0028] Establish source edges between classroom event nodes and evidence capsule nodes, establish dimension labeling edges between evidence capsule nodes and evaluation dimension nodes, establish rubric decomposition edges between evaluation dimension nodes and sub-indicator nodes, establish stage affiliation edges between classroom event nodes and classroom stage nodes, and establish participation edges between behavioral subject nodes and classroom event nodes.

[0029] The dimension-labeled edges are generated based on the evaluation dimension association information, and each edge records the time range, source modality, confidence field, and backlink field.

[0030] Optionally, S3 includes:

[0031] The evaluation rubric is broken down into sub-indicators, including: breaking down at least one evaluation dimension from classroom organization, teacher-student interaction, cognitive activation, learning feedback, and goal achievement into observation objects, behavior types, classroom stages, evidence modalities, and expected number of evidence.

[0032] Generate graph query conditions from the split results;

[0033] Candidate evidence subgraphs are retrieved based on the edge relationships between evaluation dimension nodes, sub-indicator nodes, evidence capsule nodes, and event nodes, and the dimensional relevance, cross-modal alignment score, temporal position, and modal confidence of each candidate evidence capsule are retained.

[0034] Furthermore, selecting the evidence set includes: under the premise that each sub-index query corresponds to at least one evidence capsule number and the preset time coverage condition is met, determining the dimensional relevance based on the matching degree between the candidate evidence capsule and the sub-index query;

[0035] The cross-modal consistency measure is determined based on the cross-modal alignment scores of evidence from different modalities in the same classroom event;

[0036] The time coverage is determined by querying the number of specified classroom stages, the satisfaction of the expected number of evidence, and the duration of the event based on the sub-indicators covered by the candidate evidence capsule;

[0037] The amount of evidence redundancy is determined based on text similarity, temporal overlap, and duplicate relationships identified by backlinks in the source data;

[0038] The uncertainty is determined based on the complementary values ​​of modal confidence and cross-modal alignment scores;

[0039] Using evidence redundancy and uncertainty as penalty terms, the candidate evidence capsules are sorted or screened and the evidence set is formed by increasing dimensional relevance, cross-modal consistency and time coverage according to preset weights and decreasing the penalty terms.

[0040] The preset time coverage condition is that the evidence set covers the specified classroom stage in the sub-index query, and satisfies at least one of the preset classroom stage coverage quantity, expected evidence quantity, and cumulative event duration threshold.

[0041] Optionally, S4 includes:

[0042] The evaluation dimension association information, classroom stage labels, subject identifiers, speech transcription segment features, graph adjacency features generated by the node types and edge types adjacent to the evidence capsule nodes in the classroom teaching evaluation graph, modal confidence and cross-modal alignment scores of each evidence capsule in the evidence set are input into the discriminative scoring model, and the candidate scores and candidate levels of each evaluation dimension are output.

[0043] Based on at least one type of sample, including historical expert evaluation samples and manually reviewed samples containing expert evaluation scores or manually confirmed evaluation scores, the distribution of scoring deviations between candidate scores and corresponding benchmark evaluation scores is statistically analyzed according to the evaluation dimensions.

[0044] The scoring interval is generated based on the preset quantile value of the scoring deviation distribution;

[0045] Confidence levels are generated based on the scoring interval width and modal confidence.

[0046] The review risk level is generated based on the confidence level, the width of the scoring interval, and the threshold for triggering manual review.

[0047] Optionally, S5 includes:

[0048] Generating explanatory feedback according to the controlled feedback generation template includes: writing the candidate level, scoring range, confidence level, review risk level, classroom context summary generated from the classroom context, and corresponding evidence capsule number for each evaluation dimension into the prompt template;

[0049] The constrained multimodal large model generates four fields for each feedback: evaluation conclusion, evidence capsule number, cause explanation, and actionable recommendation.

[0050] The evidence capsule number field is limited to those from the evidence set;

[0051] The executable suggestion field is limited to suggestion text that includes the target of the teaching behavior, the adjustment action, the applicable classroom stage, and the expected observation signals;

[0052] Furthermore, the verification of evidence backlink, dimensional consistency, and executability of suggestions includes: verifying that each evidence capsule number in the explanatory feedback exists in the evidence set and can be backlinked to the source data fragment through the source data backlink identifier;

[0053] Verify whether the evaluation dimensions cited in the evaluation conclusion are consistent with the evaluation dimension association information, candidate evaluation results, and scoring range of the corresponding evidence capsule;

[0054] Verify that the actionable recommendations include the target audience for the instructional behavior, the adjusted actions, the applicable classroom stage, and the expected observation signals;

[0055] When the evidence back-chain verification fails or the risk level of review reaches the preset manual review level, the corresponding evaluation dimension will be marked as manual review.

[0056] When the dimension consistency check or the suggestion executability check fails and the review risk level does not reach the preset manual review level, the corresponding evaluation dimension will be marked as regenerated or downgraded output. The downgraded output is the output of the evaluation conclusion, score range, evidence capsule number and review prompt that have passed the evidence backlink check, and the executable suggestion field that failed the check is hidden.

[0057] On the other hand, the present invention also provides a classroom teaching feedback and evaluation system based on a multimodal large model, including:

[0058] The data processing module acquires multimodal classroom data and generates classroom event sequences and classroom contexts; the capsule and graph construction module converts classroom events into evidence capsules and constructs a classroom teaching evaluation graph; the evidence retrieval module disassembles evaluation rubrics, retrieves candidate evidence subgraphs, and selects the evidence set; the scoring calibration module generates candidate evaluation results using a discriminative scoring model and outputs scoring intervals, confidence levels, and review risk levels; and the feedback verification output module calls a multimodal large model to generate explanatory feedback, performs evidence backlink verification, dimensional consistency verification, and suggestion executability verification on the explanatory feedback, and outputs at least one of the following: real-time classroom prompts and post-class comprehensive evaluation.

[0059] The beneficial effects of this invention are:

[0060] 1. By converting classroom events into evidence capsules containing time location, modality source, subject identifier, evaluation dimension association information, confidence and alignment information, source data backlink identifier, and evidence capsule number, and establishing the relationship between events, evidence capsules, and evaluation dimensions in the classroom teaching evaluation map, the evaluation conclusions can be linked back to specific source data fragments, reducing the risk of uninterpretable conclusions based solely on overall summaries.

[0061] 2. By decomposing the evaluation metric into sub-index queries, and selecting an evidence set under the constraints of at least one evidence capsule number corresponding to each sub-index query and satisfying preset time coverage conditions, the impact of redundant evidence and cross-modal alignment error on the evaluation output is reduced by combining dimensional relevance, cross-modal consistency, time coverage, and penalties for evidence redundancy and uncertainty.

[0062] 3. The candidate evaluation results are output through a discriminative scoring model and uncertainty calibration is performed. Then, the output of the multimodal large model is constrained by the template generated by controlled feedback and post-validation. This can trigger regeneration, downgraded output or manual review when the risk is high or the validation fails, thereby improving the stability and executability of the feedback suggestions. Attached Figure Description

[0063] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0064] Figure 1 This is a flowchart of a classroom teaching feedback and evaluation method based on a multimodal large model.

[0065] Figure 2 This is a flowchart of step S3 of the present invention. Detailed Implementation

[0066] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0067] refer to Figures 1-2 Classroom teaching feedback and evaluation methods based on multimodal large models include:

[0068] S1. Acquire classroom multimodal data, perform time synchronization, privacy desensitization, classroom event parsing, and classroom stage identification on the classroom multimodal data, and generate classroom event sequences and classroom contexts derived from the classroom event sequences;

[0069] S2 converts classroom events in the classroom event sequence into evidence capsules, and constructs a classroom teaching evaluation map based on classroom events, evidence capsules, evaluation dimensions, and classroom stages. The evidence capsules include time location, modality source, subject identifier, evaluation dimension association information, confidence and alignment information, source data backlink identifier, and evidence capsule number.

[0070] S3, for the preset evaluation dimensions, the evaluation rubric is broken down into sub-index queries. Relevant event nodes, evidence capsule nodes and related edges are retrieved in the classroom teaching evaluation graph to form a candidate evidence subgraph. Under the constraint that each sub-index query corresponds to at least one evidence capsule number and meets the preset time coverage condition, the evidence set is selected.

[0071] S4: Input the evidence set into the discriminant scoring model to generate candidate evaluation results, and perform uncertainty calibration on the candidate evaluation results, outputting calibration results including scoring interval, confidence level and review risk level;

[0072] S5 inputs the evidence set, candidate evaluation results, calibration results, and classroom context into the multimodal large model. It generates explanatory feedback containing evaluation conclusions, evidence capsule numbers, causal explanations, and actionable suggestions according to the controlled feedback generation template. It performs evidence backlink verification, dimensional consistency verification, and suggestion actionability verification on the explanatory feedback. Based on the verification results, it outputs at least one of the following: real-time classroom prompts and post-class comprehensive evaluation. If the verification fails, it performs at least one of the following: regeneration, downgraded output, and manual review.

[0073] In this specific embodiment, S1 includes:

[0074] Classroom multimodal data consists of classroom audio, classroom video, whiteboard images, courseware pages, and interactive terminal records. All acquisition devices write their own timestamps during acquisition. Simultaneously, it records the synchronization anchor point on the unified timeline of the classroom, which uses the interactive terminal as the master clock and continuously counts in milliseconds from the start of the class.

[0075] To achieve time synchronization, a timestamp is created for each acquisition device. Unified timeline for classroom A linear mapping is used for time alignment of all data in the device, and the linear mapping satisfies the formula... ,in This represents the aligned timeline of the class. This indicates the timestamp written by the acquisition device. This indicates the clock drift correction factor of the device relative to the unified timeline of the classroom. This indicates the fixed time delay offset of the device relative to the unified timeline of the classroom;

[0076] The and The synchronization anchors are determined by the start time of the class and every 60 seconds during the class. These synchronization anchors are broadcast by the interactive terminal and recorded locally by each acquisition device. The interactive terminal simultaneously records the value corresponding to the anchor point. Value, for multiple sets of anchor points of the same device The least squares fitting method was used to obtain and The device's audio frames, video frames, whiteboard image frames, courseware page switching frames, and interactive terminal recordings were all converted to a unified format. This ensures that different modalities have consistent time positioning on the same unified timeline of the same classroom; when a device has a missing anchor point, the device's most recent valid anchor point is used. and And it remains unchanged within the missing interval to ensure that the alignment result is reproducible and can be used for subsequent event parsing and backlink location;

[0077] After time alignment is completed, privacy desensitization is performed on the classroom multimodal data. The desensitization objects include teacher names, student names, facial regions, student IDs, and account identifiers. The desensitization performs deterministic replacement of identifiable information without disrupting the association between event analysis and subject identifiers.

[0078] In text-based data, named entity recognition is performed on the speech-to-text obtained from speech recognition and the text fields from the interactive terminal. The named entity recognition uses a sequence labeling model based on Chinese BERT, with a fixed 12-layer Transformer encoder, 768 hidden layers, 12 attention heads, a maximum sequence length of 256, and the BIO annotation system. The model output entities for personal names and student IDs are replaced with standardized placeholders, and subject identifiers are generated according to their order of appearance. The subject identifier for teachers is fixed as "Teacher," and the subject identifier for students is fixed as "Student." ",in The student is assigned a sequential number based on their first appearance in the lesson; in video data, face detection is performed frame-by-frame on the class video, and the detected face regions are Gaussian blurred. The face detection uses a RetinaFace network with a fixed input resolution. The Gaussian blur uses a confidence threshold of 0.8 and a non-maximum suppression threshold of 0.4. A Gaussian kernel with a kernel size and a standard deviation of 8 is used to ensure irreversible identification.

[0079] When performing desensitization and replacement on student ID and account identifier, a desensitization mapping table is generated and a desensitization mapping version number is written. The desensitization mapping version number is incremented by 1 each time the desensitization rule or mapping table is updated and is saved together with the classroom event sequence of this class to ensure that the consistency of the subject can be restored according to the version during subsequent review without exposing the original privacy fields.

[0080] After completing time synchronization and privacy desensitization, classroom multimodal data is analyzed to generate classroom event sequences. Classroom event analysis uses "speaking rounds, silent intervals, changes in blackboard writing, changes in courseware pages, and interactive terminal trigger records" as boundary signals and is uniformly segmented on a unified classroom timeline.

[0081] The speaking rounds are determined by speech activity detection and speaker separation. Speech activity detection uses WebRTCVAD with a fixed frame length of 30ms and a sensitivity level of 2. When the duration of a continuous silence interval reaches 1.2s, adjacent speech segments are divided into different speaking rounds.

[0082] Speaker separation is adopted - Vector embedding and hierarchical clustering with fixed embedding dimension of 256 and cosine distance threshold of 0.35; teacher speaker clusters are determined by teacher registered voices 30 seconds before the start of class and used for subsequent subject identifier alignment.

[0083] The changes in the blackboard writing are achieved by calculating the structural similarity (SSIM) of the aligned blackboard writing image frame sequence. When the SSIM of two adjacent frames is less than 0.92, the moment of the blackboard writing change is recorded and used as the event boundary.

[0084] Changes to the courseware page are determined by changes in the page index of the captured frames, and the moment of the page index change is taken as the event boundary.

[0085] Interactive terminal trigger records include operation logs such as check-in, answering questions, submission, voting, answering questions, and grading. These logs carry a unified timeline when they are written. And directly serve as the event boundary;

[0086] All the above boundary moments are merged and sorted on the unified timeline of the classroom. A classroom event is formed between adjacent boundaries. For each classroom event, the start and end times, the set of event trigger sources, the set of subject identifiers, and the indexes of speech transcription segments, video frames, whiteboard frames, courseware pages, and interactive terminal records that overlap with the time period are recorded. This results in a classroom event sequence that is arranged in chronological order and can be traced back to the anonymized source data segments.

[0087] After generating the classroom event sequence, classroom stage identification is performed on each classroom event, and classroom stage labels are generated. The set of classroom stage labels is fixed as "classroom introduction, knowledge instruction, question and interaction, practice feedback, and classroom summary". Classroom stage identification takes classroom events as the smallest labeling unit and outputs the stage label of each event. Classroom stage identification adopts a sequence classification model and utilizes the temporal correlation between events. The sequence classification model is fixed as a 2-layer Transformer encoder, a hidden layer dimension of 256, an attention head of 4, a feedforward network dimension of 1024, dropout of 0.1, and a maximum event sequence length of 128. The model input is the feature vector of each classroom event, which is composed of the event time order. The feature vector is composed of the proportion of teacher voice, the proportion of student voice, question trigger markers, interactive terminal trigger type distribution, courseware page change markers, blackboard writing change markers, and the proportion of silence, and all of them are statistically obtained within the aligned event start and end time.

[0088] The model output is the probability distribution of five types of stage labels, and the stage label corresponding to the highest probability is used as the stage label of the classroom event. At the same time, the stage label of the classroom event is written into the classroom event sequence to ensure that the subsequent evaluation dimension retrieval can be constrained by stage evidence coverage.

[0089] Classroom context is generated based on classroom event sequences, classroom stage tags, anonymized courseware page information, and interactive terminal records. The classroom context is stored in structured data and includes three parts: classroom metadata, stage statistics, and event index information. The classroom metadata consists of the course name, teaching chapter, and teaching date extracted from the courseware homepage, and all of them are anonymized text. The stage statistics consist of the cumulative event duration and number of events corresponding to each classroom stage tag. The event index information is a set of the start and end time, stage tag, main identifier, courseware page index, whiteboard frame index, and interactive terminal record index for each event in the classroom event sequence.

[0090] Simultaneously, a classroom context summary for subsequent controlled feedback is generated based on the structured classroom context. This classroom context summary is generated from a fixed template and only references the anonymized courseware page titles, the cumulative duration of each stage, and the number of key triggers on the interactive terminal. This provides reproducible and interpretable contextual input for subsequent steps without introducing long-term contextual noise.

[0091] In this specific embodiment, S2 includes:

[0092] Based on the sequence of classroom events, each classroom event is recorded as... ,in The classroom events are sequentially numbered within the lesson, starting from 1 and incrementing. The start and end times on the unified classroom timeline are respectively denoted as... and ,in Indicates the start time of a classroom event, expressed in milliseconds. Indicates the end time of the classroom event, expressed in milliseconds;

[0093] For classroom incidents Assigning evidence capsule numbers And written into an evidence capsule, the evidence capsule being numbered Use the fixed format "EC - Course Identifier - Lesson Number Identifier" "The course identifier and lesson identifier are determined by the classroom metadata and remain unchanged within the class, thereby ensuring that the evidence capsule number is unique within the class and can be reproduced when exported across systems."

[0094] When generating the evidence capsule content, it will be linked to classroom events. time interval Overlapping multimodal source data fragments are extracted into evidence fragments and written into the time location and modality source fields respectively. The modality source field is fixed to take at least one of "classroom audio, classroom video, blackboard image, courseware page, interactive terminal record" and each of them carries a corresponding source data index.

[0095] Evidence segments from classroom audio were extracted based on speech recognition results. The corresponding speech-to-text segments are written into an evidence capsule. The speech recognition model adopts an end-to-end Conformer encoder and Transformer decoder structure with fixed parameters. The encoder has 12 layers with a hidden dimension of 256 per layer and uses 4-head self-attention. The convolutional kernel size is 31 and the number of channels in the convolutional module is 256. The decoder has 6 layers with a hidden dimension of 256 and uses 4-head self-attention. The vocabulary consists of a set of Chinese characters and commonly used punctuation marks constructed based on teaching scenarios, with a vocabulary size of 6000. Decoding uses beam search with a beam width of 8 and outputs the posterior probability of each character to form a speech-to-text confidence value. For the evidence segments of the classroom video, the relevant data is extracted from the video stream. The corresponding video keyframes are written into the evidence capsule. The keyframe extraction adopts deterministic rules. Within the event interval, frames are extracted at 2fps to obtain a set of candidate frames. The color histogram difference between adjacent candidate frames is calculated. When the difference exceeds the threshold of 0.35, the candidate frame is marked as a scene switching frame and selected first. If there is no scene switching frame within the event interval, the frame corresponding to the midpoint of the event interval is selected as the keyframe, thereby ensuring that each classroom event generates at least one keyframe and can be reproduced according to a unified rule.

[0096] Evidence fragments from blackboard images and courseware pages, location and classroom events. The corresponding regions are written into the evidence capsule. The positioning adopts deterministic change region calculation. For the blackboard image frame, the first frame near the start time of the event and the last frame near the end time of the event are taken. The gray-level difference map is calculated and binarized and morphological closing operation is performed. The bounding rectangle of the connected component with the largest connected component area is taken as the blackboard change region and its pixel coordinates are recorded. For the courseware page, the new page when the page index changes within the event interval is taken as the courseware page evidence, and the preset layout frame of the page title area and the body text area is written into the region coordinates, so that the evidence areas of the blackboard and the courseware have verifiable spatial positioning.

[0097] The subject identifier field is directly written into the subject identifier set generated in step S1 and kept consistent with the desensitization mapping version number, thereby ensuring that subject drift will not occur due to desensitization when searching by subject dimension in the future.

[0098] The confidence and alignment information fields include modal confidence and cross-modal alignment score. Modal confidence is determined by speech-to-text confidence values ​​in the audio modality, by face detection visibility and occlusion ratio in the video modality, and by the stability of connected components in the changing region and image sharpness in the whiteboard and courseware modality. All of these indicators are first normalized to... The modal confidence scores are then weighted according to fixed weights and stored along with the evidence capsule to support subsequent uncertainty calculations. The cross-modal alignment score is used to measure the consistency of different modalities of evidence within the same classroom event and is also written into the evidence capsule. The cross-modal alignment score is denoted as... And calculate according to the following formula:

[0099] ;

[0100] in Represents the cross-modal alignment score and its value range is . Indicates the time alignment weight and takes the value of Represents the semantic alignment weight and takes the value of Represents the subject consistency weight and takes the value of It represents the proportion of time overlap and is obtained by the ratio of the overlap duration of different modal evidence fragments on a unified classroom timeline to the event duration. The semantic matching score is calculated by cosine similarity between the speech-transcribed segment and the interactive terminal text within the same event after encoding them using Chinese RoBERTa sentence vectors. The Chinese RoBERTa sentence vector encoder is fixed at 12 Transformer layers, with a hidden dimension of 768 and 12 attention heads, and uses the mean pooling of the last layer's token vectors to obtain the sentence vectors. The value is 1 if the subject identifier of the speaker cluster in the speech transcription segment is consistent with the subject identifier of the interactive terminal operator, and 0 otherwise.

[0101] The source data backlink identifier is used to link the evidence capsule back to the de-identified source data fragment and support integrity verification. The source data backlink identifier calculates a SHA-256 hash for each modal evidence fragment and then compares the hash hexadecimal string with the source data index of that fragment. and The evidence capsule is written together, allowing reviewers to replay fragments and verify hash consistency within the same index and time range.

[0102] The evaluation dimension association information is used to characterize the association relationship between the evidence capsule and the preset evaluation dimension metric and is written into the evidence capsule. The evaluation dimension metric is stored in a tree structure of evaluation dimensions and sub-indicators, and each sub-indicator contains three types of constraints: observation object, behavior type, and applicable classroom stage. The evaluation dimension association information is generated by a multi-label classification model. The multi-label classification model is fixed as a Chinese BERT encoder plus a fully connected Sigmoid output layer. The encoder is a 12-layer Transformer with 768 hidden dimensions. The input is a structured text concatenated from speech transcription fragments, interactive terminal text, classroom stage labels, and subject identifiers. The output is the association probability of each evaluation dimension, and a threshold of 0.6 is used to determine whether an association is established. At the same time, the list of matched evaluation dimension identifiers and their corresponding probabilities are written into the evaluation dimension association information.

[0103] After converting all classroom events into evidence capsules, a classroom teaching evaluation graph is constructed. This graph adopts an attribute graph structure and includes classroom event nodes, evidence capsule nodes, evaluation dimension nodes, sub-indicator nodes, classroom stage nodes, and behavioral subject nodes. Classroom event nodes are categorized as follows: Use as a unique identifier and record With the set of event trigger sources, the evidence capsule node is... To uniquely identify and record all fields of the evidence capsule, the evaluation dimension node uses the evaluation dimension name as the unique identifier and records the metric version number; the sub-indicator node uses "evaluation dimension name - sub-indicator number" as the unique identifier and records the observation object, behavior type, applicable classroom stage and expected number of evidence; the classroom stage node uses the stage label as the unique identifier and records the stage order constraint; and the behavior subject node uses the subject identifier as the unique identifier and records the desensitization mapping version number.

[0104] A source edge is established between classroom event nodes and evidence capsule nodes, and the time range, source modality, modality confidence, and source data backlink identifier are recorded in the edge attributes. A dimension labeling edge is established between evidence capsule nodes and evaluation dimension nodes, and the evaluation dimension association probability, time range, source modality, modality confidence, cross-modality alignment score, and source data backlink identifier are recorded in the edge attributes. A metric decomposition edge is established between evaluation dimension nodes and sub-index nodes, and the sub-index constraint field is recorded in the edge attributes. A stage affiliation edge is established between classroom event nodes and classroom stage nodes, and the stage label source is recorded as the classroom stage identification model in the edge attributes. A participation edge is established between behavioral subject nodes and classroom event nodes, and the role type and participation evidence source of the subject in the event are recorded in the edge attributes. Thus, a classroom teaching evaluation graph that can be queried by nodes and edge attributes and can link the evaluation conclusion back to the evidence capsule and further back to the source data fragment is obtained.

[0105] In this specific embodiment, S3 includes:

[0106] The system reads the version number of the evaluation rubric corresponding to the preset evaluation dimension set of "classroom organization, teacher-student interaction, cognitive activation, learning feedback, and goal achievement" for this lesson. Starting from the evaluation dimension nodes in the classroom teaching evaluation graph, it obtains all sub-indicator nodes along the rubric decomposition edges and identifies each sub-indicator node as a sub-indicator query. And it is solidified in the form of a structured quintuple:

[0107] ;

[0108] in Indicates the first Individual indicator query This indicates the object of observation and its value can be either the teacher, the student, or the whole class. This indicates the behavior type and its value is one of questioning, lecturing, feedback, organizing, or summarizing, and it must be consistent with the metric field. This represents a subset of the set of tags applicable to different classroom stages, with values ​​of "classroom introduction, knowledge instruction, questioning and interaction, practice feedback, and classroom summary". This represents a subset of the evidence modalities, whose values ​​are "classroom audio, classroom video, blackboard images, courseware pages, and interactive terminal records". This indicates the desired number of pieces of evidence and is stored as 1, 2, or 3 in the gauging and used as a quantity constraint in subsequent selections;

[0109] Query for each sub-indicator In the classroom teaching evaluation graph, a graph query is performed to form a candidate evidence subgraph. The graph query uses attribute graph matching and starts from the sub-indicator node to retrieve evidence capsule nodes in reverse along the dimension label edges. Then, it associates classroom event nodes along the source edge and classroom stage nodes along the stage affiliation edge, while also associating behavioral subject nodes along the participation edge. The filtering condition of the graph query is fixed as the stage label of the classroom stage node belonging to... Furthermore, the modality source field of the evidence capsule node is... There is an intersection and the behavioral subject node and the observed object Consistency, where the consistency between the actor node and the observed object is determined by rules, when When the subject is a teacher, retain the evidence capsule with the subject identifier of the teacher. When identifying a student, retain the student identifier. Evidence capsule, when A capsule that preserves the primary identifiers of both teachers and students simultaneously throughout the class;

[0110] For each candidate evidence capsule returned by the graph query The evidence capsule's ID, time location, modality source, and subject identifier are retained. The evaluation dimension association probability is read from the dimension annotation edges as the base score for the capsule's dimension association. Simultaneously, the modality confidence score and cross-modality alignment score are read from the evidence capsule's confidence and alignment information fields. The start and end times of the event are read from the classroom event nodes to obtain the event duration and time location, thus enabling the determination of each... A candidate evidence subgraph containing event nodes, evidence capsule nodes, and associated edges is generated, and an inverted index is built in memory for "sub-index query". Candidate evidence capsule set ";

[0111] When selecting the evidence set, the system satisfies the query requirements of each sub-indicator. Under the constraint of corresponding to at least one evidence capsule number and satisfying a preset time coverage condition, a deterministic two-stage selection process is executed. The first stage involves each... from Select one evidence capsule to add to the evidence set. To satisfy the coverage constraint of "at least one", the selection process involves sequentially trying and skipping candidate evidence capsules that conflict with already selected evidence capsules based on their comprehensive scores from high to low. The redundancy conflict is determined according to rules. If the source data backlink identifier of any of the candidate evidence capsules is the same, it is considered a redundant conflict. If the cosine similarity of any speech transcription segment of any evidence capsule is greater than 0.92 and the time overlap ratio of the two on the unified timeline of the classroom is greater than 0.8, it is judged as a redundant conflict. The Chinese RoBERTa sentence vector encoder is fixed as a 12-layer Transformer, a hidden dimension of 768, and a number of attention heads of 12, and the sentence vector is obtained by mean pooling of the last layer of token vector.

[0112] The second phase was obtained from the first phase. Based on the expected number of evidence For each Continue to replenish the evidence capsules until each The quantity constraint is satisfied, and a preset time coverage condition is also enforced, wherein the preset time coverage condition is fixed as the evidence set. The query covers all classroom stage tags that appear in all sub-index queries, and the number of different classroom stage tags covered is equal to 4. The corresponding cumulative event duration is not less than 900 seconds, where the cumulative event duration is determined by... The classroom event time intervals corresponding to each evidence capsule are sorted by start time, overlapping intervals are merged, and the lengths of the merged intervals are summed to obtain the result, thereby avoiding the same time period being counted repeatedly.

[0113] To achieve a reproducible comprehensive scoring and ranking, the system performs a multi-stage evaluation of each candidate evidence capsule. Calculate the overall score And accordingly in each The formula for calculating the overall score based on the internal ranking is as follows:

[0114] ;

[0115] in Indicating candidate evidence capsules The overall score and the range of values ​​is This represents the dimensional relevance weight and takes values ​​of 1 to 2. Represents the cross-modal consistency weight and takes the value of Indicates the time coverage weight and takes the value of This represents the weight of the evidence redundancy penalty and its value is... This represents the penalty weight for uncertainty and takes the value of . The evaluation dimension correlation probability and sub-index query represent the dimensional relevance and are defined by the dimension-labeled edges. The object of observation Behavioral types Applicable to classroom stages Evidence modality The matching results are jointly determined and normalized to This represents the cross-modal consistency measure, directly derived from and normalized to the cross-modal alignment score in the evidence capsule. The time coverage is represented by the contribution of the duration of the corresponding classroom event in the evidence capsule and its classroom stage label to the currently unmet stage coverage needs, and is normalized to [a specific value]. Represents the redundancy of evidence and is determined by the candidate evidence capsule and the current evidence set. The results of redundancy determination based on three categories—duplicate source data backlink identifiers, semantic similarity of speech-transcribed segments, and temporal overlap—were summarized and normalized. The uncertainty is represented by the complementarity of the modal confidence and cross-modal alignment scores of the evidence capsule and is normalized to [value missing]. ;

[0116] The system outputs a set of evidence based on the above two-stage process and the comprehensive scoring and ranking results. And simultaneously output the query results for each sub-indicator. The corresponding list of selected evidence capsule numbers, the set of classroom stage labels covered by the evidence set, and the cumulative event duration are used as inputs to the discriminative scoring model in S4 and as constraints on the evidence capsule number field in the controlled feedback generation template in step S5.

[0117] In this specific embodiment, S4 includes:

[0118] The system records the evidence set as And it is collected in the classroom teaching evaluation map according to the evaluation dimensions, and each evaluation dimension is recorded as follows: ,in Indicates the evaluation dimension index and The value can be one of "classroom organization, teacher-student interaction, cognitive activation, learning feedback, or goal achievement";

[0119] The system is Each evidence capsule node reads its evaluation dimension association information, classroom stage label, subject identifier, speech transcription segment features, graph adjacency features, modal confidence and cross-modal alignment score, and constructs the input vector of the discriminative scoring model. The speech transcription segment features are generated by a fixed Chinese RoBERTa sentence vector encoder, which is a 12-layer Transformer with a hidden dimension of 768, 12 attention heads, and uses mean pooling of the last layer token vector to obtain a 768-dimensional vector.

[0120] The graph adjacency characteristics are obtained by statistically analyzing the node types and edge types that are one hop adjacent to the evidence capsule node in the classroom teaching evaluation graph. The node types are fixed as classroom event nodes, evaluation dimension nodes, classroom stage nodes, and behavioral subject nodes. The number of adjacent nodes of each type is counted and normalized. The edge types are fixed as source edges, dimension labeling edges, stage attribution edges, and participation edges. The number of adjacent edges of each type is counted and normalized. The system then concatenates the statistical results in a fixed order to form an adjacency feature vector; after encoding the evidence capsule into a vector, the system sorts it according to the evaluation dimensions. Aggregating the evidence capsule vectors belonging to this dimension yields dimensional features, which are then input into a discriminative scoring model to output candidate scores and candidate levels. The aggregation employs a deterministic weighted average, with weights determined by the corresponding information of the evidence capsules in the evaluation dimension's association information. The correlation probability is normalized.

[0121] The discriminant scoring model is fixed as a multilayer perceptron structure with fixed parameters, and the input layer dimension is equal to the "speech-to-text segment feature dimension". Adjacency feature dimension The scalar feature dimension 6 consists of the one-hot index mapping value of the classroom stage label, the one-hot index mapping value of the subject identifier, the modal confidence, the cross-modal alignment score, the evidence capsule duration, and the number of event trigger sources.

[0122] The multilayer perceptron contains three fully connected hidden layers, and the number of neurons in each hidden layer is as follows: The activation function is fixed at ReLU and the dropout is fixed at 0.2. The output contains two parallel headers, with the score header outputting the candidate scores. And the output range is limited to This is achieved through linear scaling after Sigmoid mapping, with candidate levels output in the level header. And the level set is fixed as And obtain it by taking the class with the maximum probability using Softmax;

[0123] A deterministic correspondence is established between candidate levels and candidate scores for consistency constraints, with the level threshold fixed at [value missing]. correspond , correspond correspond correspond correspond ;

[0124] After obtaining the candidate scores for each evaluation dimension, uncertainty calibration is performed. The system reads historical expert evaluation samples and manual review samples to form a calibration set and then categorizes them by evaluation dimension. The distribution of scoring bias was statistically analyzed separately. Each sample in the calibration set contained candidate model scores under the same set of input features. Compared with the benchmark score given by humans ,in This represents the benchmark evaluation score determined by expert evaluation or manually confirmed evaluation scores, and its range is [value missing]. And calculate the scoring deviation. Later Establish experience distributions according to each dimension;

[0125] The system reads quantiles from the empirical distribution at preset quantile values ​​of 0.1 and 0.9. and And generate a rating range Its calculation formula is ,in Indicate evaluation dimensions The lower bound of the scoring interval is calculated and then truncated to... , Indicate evaluation dimensions The upper bound of the scoring interval is calculated and then truncated to... This represents the candidate score for that dimension. Indicates scoring bias The value at the 0.1 quantile. Indicates scoring bias The value at the 0.9 quantile;

[0126] The system uses the width of the scoring interval The average modal confidence of the evidence capsule in this dimension Together, confidence levels are generated, the The modal confidence average of the selected evidence capsules for this dimension, with values ​​ranging from [value range missing]. The confidence level set is fixed as follows High, Medium, Low And the judgment rule is fixed as follows and When it is high, and Furthermore, if the condition is not high, it is considered medium; otherwise, it is considered low.

[0127] The system then bases its decisions on confidence level and scoring interval width. The review risk level is generated by triggering a threshold for manual review, and the set of review risk levels is fixed. High risk, medium risk, low risk And the threshold for triggering manual review is fixed at 1. Or a high risk is triggered when the confidence level is low, and a high risk is not triggered and The current situation is classified as medium risk; the rest are low risk.

[0128] The system ultimately outputs each evaluation dimension. Corresponding candidate evaluation results With calibration results (Confidence level, review risk level) and together with the evidence set The list of evidence capsule numbers will be included in subsequent steps. The controlled feedback generation template input supports generation by evidence scope and triggering of review by risk level.

[0129] In this specific embodiment, S5 includes:

[0130] The system will collect evidence. Evaluation Dimensions Corresponding candidate evaluation results With calibration results The confidence level and review risk level, along with the classroom context and classroom context summary generated by S1, are jointly organized into a controlled feedback generation template and input into the multimodal large model. To generate explanatory feedback, in which This indicates that the lesson has been selected for the evaluation of the set of evidence capsules, and each evidence capsule carries a unique evidence capsule number. Source data backlink identifier and modal fragment index, This indicates the evaluation dimensions, with values ​​limited to "classroom organization, teacher-student interaction, cognitive activation, learning feedback, and goal achievement". This indicates the discriminative scoring model for evaluation dimensions. Output candidate scores and their range is Indicates the candidate level and takes the value and These represent the lower and upper bounds of the scoring interval, respectively, and their values ​​range from 1 to 2. Multimodal large model Inference is performed using a deterministic concatenated structure of a visual encoder and a language model, with parameters frozen. The visual encoder employs a ViT-L / 14 architecture and has a fixed input resolution. It outputs a visual token sequence. The language model adopts a decoding Transformer structure with a fixed number of 32 layers, a fixed hidden dimension of 4096, a fixed number of attention heads of 32, and a fixed maximum context length of 8192 tokens. Cross-modal fusion is achieved by concatenating the visual token as a prefix token before the text token. The inference parameters are fixed at temperature of 0.2, top_p of 0.9, and a maximum generation length of 1200 tokens. External retrieval tools are disabled to ensure the output is reproducible.

[0131] The controlled feedback generation template is populated line by line at the dimensional granularity and uses a strongly constrained JSON output format. The template includes parameters for each evaluation dimension. Write to candidate level Scoring range Confidence level, review risk level, classroom context summary, and a whitelist of evidence capsule numbers related to this dimension. ,in Indicates from The filtered evidence capsule set contains only the evaluation dimension-related information. Evidence capsule number;

[0132] The template imposes hard constraints on the fields of the explanatory feedback and requires... For each Generate one and only four fields: "Evaluation Conclusion", "Evidence Capsule Number", "Cause Explanation", and "Executable Recommendation". The "Evidence Capsule Number" field is constrained to be available only from a whitelist. Select and return at least one evidence capsule number. The "Executable Suggestion" field is constrained to include four subfields: teaching behavior object, adjustment action, applicable classroom stage, and expected observation signal, and output with fixed labels: "object, action, stage, signal".

[0133] The system provides the speech-to-text segments from the evidence capsule, the text recorded on the interactive terminal, keyframe images, and the coordinates of the whiteboard and courseware areas as multimodal inputs. And use the evidence capsule number in the template Establish citation relationships to ensure that specific fragments can be cited within the scope of controlled evidence when the model generates explanations of causes;

[0134] After the explanatory feedback is generated, the system performs three types of post-validation and outputs real-time classroom prompts and post-class comprehensive evaluations based on the validation results. The evidence backlink validation assigns a number to each evidence capsule returned in each feedback. Perform a three-step verification, the first step is verification Does it belong to the whitelist for the corresponding dimension? And whether it belongs to a collection of evidence The second step is to locate the classroom teaching evaluation map. The corresponding evidence capsule node is read and its source data backlink identifier and source data index are read. The third step is to retrieve the desensitized source data fragment based on the source data index and the time recorded in the evidence capsule, and recalculate the SHA-256 hash to compare it byte by byte with the source data backlink identifier to confirm the backlink consistency.

[0135] Dimensional consistency verification performs dimension alignment verification on each feedback and requires that the evaluation dimension corresponding to the evaluation conclusion must be equal to the dimension of the current processing. And all the feedback referenced in this message In the evaluation dimensions of the evidence capsule, related information is used for... The correlation probability is not less than 0.6, and the rating description of the evaluation conclusion must be consistent with the candidate level. Consistent and after mapping the conclusion text to level labels, with The comparison implementation uses a fixed mapping rule: conclusions containing "needs improvement" are mapped to 1 or 2, those containing "meets requirements" are mapped to 3, those containing "good performance" are mapped to 4, and those containing "outstanding performance" are mapped to 5. Coverage is required when the mapping result is a range. ;

[0136] It is recommended that the executability check perform structured integrity verification on the "executable suggestion" field and require the simultaneous presence of four tags: "object, action, stage, and signal," with each tag's content length not less than 4 Chinese characters. Furthermore, the "stage" tag content must belong to one of the classroom stage tag sets: "classroom introduction, knowledge instruction, question and interaction, practice feedback, and classroom summary," thereby ensuring that the suggestion can be implemented.

[0137] When the evidence backlink verification fails or the review risk level for that evaluation dimension is marked as high risk, the system will mark the corresponding evaluation dimension as requiring manual review and output the following: The task of reviewing the list of cited evidence capsule numbers and the reasons for backlink failure is performed. At the same time, the automatic output side only retains the evaluation conclusions and evidence capsule numbers that pass the backlink verification, along with additional review prompts.

[0138] When the dimensional consistency check or the suggested executability check fails and the review risk level is not high risk, the system will regenerate or downgrade the output for that evaluation dimension. Regeneration is always performed twice, and each time the failed check type and the location of the violated field are written into the error feedback area of ​​the template, and the temperature is set to 0 to force deterministic correction. If it still fails after two regenerations, downgraded output is performed. The downgraded output is always the evaluation conclusion and the scoring range. The evidence capsule number and review prompts are displayed, and the "Executable Suggestion" field is hidden.

[0139] In terms of output format, the system will write feedback items that meet the criteria of having a "stage" label equal to the current classroom stage and whose referenced evidence capsule time location falls within the most recent 180 seconds into the classroom instant prompt. The remaining feedback items will be summarized into the post-class comprehensive evaluation according to the evaluation dimensions, and each feedback item will be accompanied by an evidence capsule number to support the reviewers to link back to the desensitized source data fragment by number to complete traceable verification.

[0140] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0141] This invention transforms multimodal classroom data into a searchable, linkable, and verifiable evidence structure through classroom event sequences, evidence capsules, and classroom teaching evaluation maps. This enables the scoring model and the multimodal large model to generate evaluation results and feedback text within a controlled evidence range, thereby reducing the impact of long contextual noise, cross-modal alignment errors, and redundant information on classroom evaluation output.

[0142] This invention does not change the basic synergistic relationship between the evaluation graph, the discriminant scoring model, and the multimodal large model generation framework. Instead, it adds an evidence capsule structure before events enter the evaluation graph, adds evidence set selection constraints during the evidence retrieval stage, adds uncertainty calibration after scoring, and adds evidence backlinks, dimensional consistency, and suggestion executability verification after feedback generation. This better enables traceable, stable, and verifiable classroom teaching feedback and evaluation.

Claims

1. A classroom teaching feedback and evaluation method based on a multimodal large model, characterized in that, include: S1. Acquire classroom multimodal data, perform time synchronization, privacy desensitization, classroom event parsing, and classroom stage identification on the classroom multimodal data, and generate classroom event sequences and classroom contexts derived from the classroom event sequences; S2 converts classroom events in the classroom event sequence into evidence capsules, and constructs a classroom teaching evaluation map based on classroom events, evidence capsules, evaluation dimensions, and classroom stages. The evidence capsules include time location, modality source, subject identifier, evaluation dimension association information, confidence and alignment information, source data backlink identifier, and evidence capsule number. S3, for the preset evaluation dimensions, the evaluation rubric is broken down into sub-index queries. Relevant event nodes, evidence capsule nodes and related edges are retrieved in the classroom teaching evaluation graph to form a candidate evidence subgraph. Under the constraint that each sub-index query corresponds to at least one evidence capsule number and meets the preset time coverage condition, the evidence set is selected. S4: Input the evidence set into the discriminant scoring model to generate candidate evaluation results, and perform uncertainty calibration on the candidate evaluation results, outputting calibration results including scoring interval, confidence level and review risk level; S5 inputs the evidence set, candidate evaluation results, calibration results, and classroom context into the multimodal large model. It generates explanatory feedback containing evaluation conclusions, evidence capsule numbers, causal explanations, and actionable suggestions according to the controlled feedback generation template. It performs evidence backlink verification, dimensional consistency verification, and suggestion actionability verification on the explanatory feedback. Based on the verification results, it outputs at least one of the following: real-time classroom prompts and post-class comprehensive evaluation. If the verification fails, it performs at least one of the following: regeneration, downgraded output, and manual review.

2. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 1, characterized in that, S1 includes: Align classroom audio, classroom video, blackboard images, courseware pages, and interactive terminal records based on the timestamps of the acquisition devices and the unified timeline of the classroom. The system performs desensitization and replacement on teacher names, student names, facial regions, student IDs, and account identifiers while retaining the desensitization mapping version number; it categorizes classroom events based on speaking rounds, silence intervals, blackboard changes, courseware page changes, and interactive terminal trigger records; it labels classroom events with classroom stage tags based on a set of stage tags for classroom introduction, knowledge delivery, Q&A interaction, practice feedback, and classroom summary; and it generates the classroom context based on the classroom event sequence, classroom stage tags, desensitized courseware page information, and interactive terminal records.

3. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 1, characterized in that, Step S2, generating the evidence capsule, includes: assigning evidence capsule numbers to classroom events; extracting speech-to-text segments corresponding to the start and end times of the events from the speech recognition results; extracting video keyframes corresponding to the start and end times of the events from the video stream; locating the regions corresponding to the classroom events from the whiteboard images and courseware pages; calculating modal confidence based on the recognizer output probability, occlusion ratio, transcription confidence value, and device synchronization deviation; calculating cross-modal alignment scores based on the time overlap ratio, semantic matching score, and subject identifier consistency; calculating hash values ​​for the source data segments and writing them into the source data backlink identifier; and generating the evaluation dimension association information based on the matching relationship between the evaluation dimension metric and the classroom event content, classroom stage labels, and subject identifiers.

4. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 1, characterized in that, Step S2, constructing the classroom teaching evaluation graph, includes: establishing classroom event nodes, evidence capsule nodes, evaluation dimension nodes, sub-indicator nodes, classroom stage nodes, and behavioral subject nodes; establishing source edges between classroom event nodes and evidence capsule nodes, establishing dimension labeling edges between evidence capsule nodes and evaluation dimension nodes, establishing rubric decomposition edges between evaluation dimension nodes and sub-indicator nodes, establishing stage attribution edges between classroom event nodes and classroom stage nodes, and establishing participation edges between behavioral subject nodes and classroom event nodes; wherein the dimension labeling edges are generated based on the evaluation dimension association information, and each edge records the time range, source modality, confidence field, and backlink field.

5. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 4, characterized in that, Step S3, which decomposes the evaluation rubric into sub-indicator queries, includes: decomposing at least one evaluation dimension among classroom organization, teacher-student interaction, cognitive activation, learning feedback, and goal achievement into observation objects, behavior types, classroom stages, evidence modalities, and expected evidence quantities; generating graph query conditions based on the decomposition results; retrieving candidate evidence subgraphs based on the edge relationships between evaluation dimension nodes, sub-indicator nodes, evidence capsule nodes, and event nodes, and retaining the dimension relevance, cross-modal alignment score, time position, and modal confidence of each candidate evidence capsule.

6. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 5, characterized in that, Step S3 involves selecting the evidence set by: determining dimensional relevance based on the matching degree between candidate evidence capsules and sub-indicator queries, provided that each sub-indicator query corresponds to at least one evidence capsule number and the preset time coverage condition is met; determining cross-modal consistency based on cross-modal alignment scores of evidence from different modalities in the same classroom event; determining time coverage based on the number of specified classroom stages covered by the candidate evidence capsules, the satisfaction of the expected number of evidence, and the event duration; determining evidence redundancy based on text similarity, time overlap, and repetition of source data backlink identifiers; determining uncertainty based on the complementary values ​​of modal confidence and cross-modal alignment scores; using evidence redundancy and uncertainty as penalty items, increasing dimensional relevance, cross-modal consistency, and time coverage according to preset weights and decreasing the penalty items, sorting or filtering candidate evidence capsules to form the evidence set; wherein the preset time coverage condition is that the evidence set covers the classroom stages specified by the sub-indicator query and satisfies at least one of the preset classroom stage coverage number, expected number of evidence, and cumulative event duration threshold.

7. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 6, characterized in that, S4 includes: inputting the evaluation dimension association information, classroom stage labels, subject identifiers, speech-to-text segment features, graph adjacency features generated from the node types and edge types adjacent to the evidence capsule nodes in the classroom teaching evaluation graph, modal confidence, and cross-modal alignment scores of each evidence capsule in the evidence set into a discriminative scoring model, and outputting candidate scores and candidate levels for each evaluation dimension; based on at least one type of sample, including historical expert evaluation samples containing expert evaluation scores or manually confirmed evaluation scores and manually reviewed samples, statistically analyzing the scoring deviation distribution between candidate scores and corresponding benchmark evaluation scores according to the evaluation dimensions; generating a scoring interval based on the preset quantile value of the scoring deviation distribution; generating a confidence level based on the scoring interval width and modal confidence; and generating a review risk level based on the confidence level, scoring interval width, and manual review trigger threshold.

8. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 7, characterized in that, Step S5, generating explanatory feedback according to the controlled feedback generation template, includes: writing the candidate level, scoring range, confidence level, review risk level, classroom context summary generated from the classroom context, and corresponding evidence capsule number for each evaluation dimension into the prompt template; constraining the multimodal large model to generate four fields for each feedback: evaluation conclusion, evidence capsule number, cause explanation, and actionable suggestion; limiting the evidence capsule number field to those from the evidence set; and limiting the actionable suggestion field to suggestion text containing the teaching behavior object, adjustment action, applicable classroom stage, and expected observation signals.

9. The classroom teaching feedback and evaluation method based on a multimodal large model according to claim 8, characterized in that, Step S5 includes verifying the evidence backlink, dimension consistency, and suggestion executability. This involves: verifying that each evidence capsule number in the explanatory feedback exists in the evidence set and can be linked back to the source data segment via the source data backlink identifier; verifying that the evaluation dimension referenced in the evaluation conclusion is consistent with the evaluation dimension association information, candidate evaluation results, and scoring range of the corresponding evidence capsule; verifying that the executable suggestion includes the teaching behavior object, adjustment action, applicable classroom stage, and expected observation signal; when the evidence backlink verification fails or the review risk level reaches the preset manual review level, the corresponding evaluation dimension is marked for manual review; when the dimension consistency verification or suggestion executability verification fails and the review risk level does not reach the preset manual review level, the corresponding evaluation dimension is marked for regeneration or downgraded output. The downgraded output is the output of the evaluation conclusion, scoring range, evidence capsule number, and review prompt that passed the evidence backlink verification, and the executable suggestion fields that failed the verification are hidden.

10. A classroom teaching feedback and evaluation system based on a multimodal large model, used to execute the classroom teaching feedback and evaluation method based on a multimodal large model as described in any one of claims 1 to 9, characterized in that, include: The data processing module is used to acquire multimodal classroom data and generate classroom event sequences and classroom contexts; The capsule and graph construction module is used to convert classroom events into evidence capsules and construct classroom teaching evaluation graphs. The evidence retrieval module is used to decompose the evaluation rubric, retrieve candidate evidence subgraphs, and select the evidence set; the scoring calibration module is used to generate candidate evaluation results through a discriminative scoring model and output the scoring range, confidence level, and review risk level. The feedback verification output module is used to call the multimodal large model to generate explanatory feedback, perform evidence backlink verification, dimensional consistency verification, and suggestion executability verification on the explanatory feedback, and output at least one of the following: real-time classroom prompts and post-class comprehensive evaluation.

Images