State-driven 5e textbook explainable evaluation system

Abstract

The present application relates to the technical field of intelligent evaluation of teaching materials, and specifically discloses a state-driven 5E teaching material explainable evaluation system, which comprises a teaching material evaluation data construction module, a text analysis module, a stage target analysis module, a logic break detection module and the like, and is used for respectively identifying key teaching elements of the teaching material, performing teaching target matching analysis, performing stage-to-stage problem chain continuity analysis, and performing defect supplement analysis; and a teaching material comprehensive evaluation report is generated according to the intermediate analysis results. The present application constructs a stage-to-stage problem chain continuity detection mechanism, analyzes the problem transmission relationship between different stages of the 5E teaching material, introduces the discovered logic break into the comprehensive scoring process, and improves the consistency between the comprehensive scoring result and the actual teaching quality.

Description

Technical Field

[0001] This invention relates to the field of intelligent evaluation technology for teaching materials, and in particular to an interpretable evaluation system for state-driven 5E teaching materials. Background Technology

[0002] The 5E teaching model comprises five stages: Excitation (arousing interest and raising questions), Exploration (using first principles to explore the essence of problems), Enhancement (learning the knowledge and skills necessary to solve problems), Execution (hands-on problem-solving), and Evaluation (evaluation and reflection). It has a clear division of stages and a logical progression. Each stage not only undertakes different teaching functions, but subsequent stages also need to be developed around the questions, goals, and tasks raised in the preceding stages.

[0003] Currently, for automatic evaluation tasks of structured text, especially those based on large language models, existing technologies mainly fall into four categories: direct assessment methods, chain-of-thought-based evaluation methods, agent-based evaluation methods, and decomposed / hierarchical evaluation methods. Among these, decomposed and agent-based methods are more suitable for multi-stage analysis. Decomposed evaluation methods break down the text structure, dividing the overall evaluation task into multiple sub-tasks, which are then analyzed and evaluated separately. Agent-based evaluation methods break down the evaluation task into multiple analysis steps and utilize mechanisms such as planning, reasoning, memory, and reflection to complete complex evaluation tasks.

[0004] However, most existing educational evaluation systems focus on the accuracy of knowledge, the quality of language, the quality of instructional design, or the evaluation of learning outcomes. The evaluation objects are usually single chapters, course designs, or teaching activities, lacking specific modeling mechanisms for the unique stage progression relationships, stage goal consistency, and cross-stage logical connections inherent in the 5E teaching model. While agent-based evaluation methods and decomposition-based evaluation methods can accomplish automatic text evaluation tasks to some extent, they have the following shortcomings when applied to 5E textbooks: (1) Lack of structured state management mechanism. Existing methods mostly use single-round reasoning or multi-round natural language interaction. Intermediate analysis results are recorded and transmitted in natural language form. There is a lack of unified structured state representation between each step, making it difficult to share the obtained analysis conclusions. This can easily lead to duplicate analysis, information omission, and reasoning drift.

[0005] (2) Lack of ability to model logical relationships across structural units. Existing methods mainly focus on the content quality of individual structural units themselves, and rarely analyze the logical dependencies and target inheritance relationships between units, making it difficult to discover problems such as logical breaks, target deviations and content disconnects in the text.

[0006] (3) Lack of an interpretable evaluation mechanism. Existing methods are mostly aimed at generating scoring results. Evaluation reasons are usually generated after scoring, lacking a clear correspondence with the original text content and a unified evidence extraction and organization mechanism, resulting in insufficient traceability of evaluation results.

[0007] (4) Lack of self-checking and supplementary discovery mechanism for evaluation results. Existing methods score based on predefined dimensions, but in actual evaluation, the text may have defects not covered by the evaluation dimensions, such as ambiguity, repetition, and unclear target expression. Existing technology is difficult to conduct a comprehensive supplementary check after the dimension scoring is completed, and it is easy to miss important issues that affect the quality of the text.

[0008] (5) Difficult to adapt to the evaluation task of structured text with stage-dependent relationships. For structured text with clear stage progression relationships, the existing technology lacks a special analysis mechanism for stage goal inheritance, functional synergy and logical closure, making it difficult to comprehensively evaluate the overall structural quality of such text. Summary of the Invention

[0009] This invention aims to solve the aforementioned problems. To this end, this invention provides a state-driven interpretable evaluation system for 5E teaching materials, constructing a cross-stage problem chain continuity detection mechanism to analyze the problem continuity relationships between different stages of the 5E teaching materials, and incorporating discovered logical breaks into the comprehensive scoring process to improve the consistency between the comprehensive scoring results and actual teaching quality.

[0010] This invention provides a state-driven interpretable evaluation system for 5E teaching materials, and the technical solution adopted is as follows: including: The textbook evaluation data construction module is used to acquire textbooks, divide the content of textbooks according to chapters and stages, add attribute annotations, and construct evaluation input objects; The text parsing module is used to perform semantic analysis on the evaluation input objects according to the preset parsing objectives of each stage, identify key teaching elements, and index the text content of the evaluation input objects corresponding to the key teaching elements to obtain the parsing results. The stage goal analysis module is used to perform teaching goal matching analysis on the analysis results and evaluation input objects based on preset stage goals, and obtain the stage goal analysis results. The logic break detection module is used to analyze the continuity of the problem chain between stages based on the parsing results and the stage target analysis results, and to obtain the logic break results. The logic break detection module analyzes the continuity of the problem chain between the previous stage and the current stage to obtain the logic break result; Among them, the problem chain continuity analysis examines the connection between analysis stages and detects whether the current stage effectively inherits the key results formed in the previous stage; it includes core problem consistency analysis, cognitive object consistency analysis, problem progression relationship analysis, and reasoning connection relationship analysis. The evidence extraction module is used to extract evidence from the evaluation input objects based on the preset evaluation dimensions, the analysis results and the stage target analysis results, construct an explicit evidence chain, and obtain an evidence pool. The defect detection module is used to determine the evaluation coverage based on the analysis results, stage target analysis results, evaluation input objects, logic break results, and evidence pool. Outside the evaluation coverage, it performs supplementary defect analysis on the content of the teaching materials and generates defect descriptions. The dimension scoring module is used to score based on the preset scoring rules corresponding to the evaluation dimensions, according to the evidence pool, stage target analysis results, logical break results, and defect descriptions, and generate dimension scoring results. The results summary module is used to generate a comprehensive evaluation report of the teaching materials based on the dimensional scoring results, logical break results, and defect descriptions.

[0011] Furthermore, the textbook evaluation data construction module divides the textbook content by chapter, and divides the content of each chapter into stages according to the five teaching stages of the 5E textbook, obtaining different textbook content fragments for each stage; add attribute labels to obtain the evaluation input objects; among which, the attribute labels include: stage labels, subject labels, and grade level labels.

[0012] Furthermore, for each stage of the evaluation input, the text parsing module determines the parsing target based on the stage label of the evaluation input; combined with the parsing target, it performs semantic analysis on the textbook content fragments of the evaluation input to identify key teaching elements; it establishes an index for the textbook content fragments corresponding to the key teaching elements; and it generates the parsing results based on the key teaching elements and their corresponding textbook content fragment indexes.

[0013] Furthermore, each stage objective includes one or more objective items, which correspond to the teaching behaviors or learning activities of the current stage; Input the stage goals, analysis results, and evaluation inputs into the large language model. For each goal item, the large language model outputs: goal name, achievement status, supporting basis, and reasons for achievement or lack thereof.

[0014] Furthermore, the evidence extraction module locates the corresponding key teaching elements from the analysis results based on the evaluation dimensions, and extracts evidence content from the textbook content fragments by combining the achievement status of the key teaching elements in the stage goal analysis results; it also summarizes all evidence content of all stages and all evaluation dimensions, establishes the correspondence between evidence content and evaluation dimensions, and constructs the evidence pool.

[0015] Furthermore, for each evaluation dimension, the dimension scoring module matches the stage target analysis results with key sub-points and non-key sub-points point by point; then, combined with the logical break results and defect descriptions, it makes a judgment on the achievement of the evaluation dimension and obtains the achievement status of the evaluation dimension; it performs graded matching on the achievement status of the evaluation dimension to generate a dimension score and obtain the dimension scoring result, which includes the dimension score and the corresponding scoring basis. Among them, the key sub-point focuses on whether the teaching functions corresponding to the current dimension are effectively implemented.

[0016] Furthermore, the results summary module calculates the stage score for each stage based on the dimensional scoring results and preset weights, and corrects the stage scores based on the logical break results and defect descriptions to obtain the comprehensive score for each stage; based on the comprehensive score, dimensional scoring results, logical break results, and defect descriptions, a comprehensive evaluation report of the teaching materials is generated.

[0017] Furthermore, the logic break detection module classifies the logic break levels based on the logic break results; The results summary module obtains the logic continuity penalty coefficient based on the logic break level and corrects the stage score. For the stage where a logic break is detected for the first time, the corresponding logic continuity penalty coefficient is determined based on the logic break level. For subsequent stages where logic breaks occur continuously, the value of the logic continuity penalty coefficient is progressively corrected to reduce its value.

[0018] Furthermore, it also includes: an evaluation status object module, used to construct a unified evaluation status object. The evaluation status object stores evaluation input objects, parsing results, stage target analysis results, logic break results, evidence pool, defect descriptions, dimension scoring results, as well as preset parsing targets, stage targets, evaluation dimensions, and scoring rules.

[0019] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: (1) This invention constructs a multi-level analysis process, including text parsing, stage target analysis, logical break detection, evidence extraction and defect detection, to form a structured evaluation state before scoring, and generates evaluation results based on unified scoring rules, thereby reducing the subjectivity and randomness in the evaluation process and improving the accuracy and consistency of textbook evaluation results.

[0020] (2) By establishing an evidence extraction mechanism and a defect attribution mechanism, this invention establishes a correspondence between the evaluation conclusion and the original content of the textbook, and can clearly point out the defect location, defect manifestation and its impact cause, so that the evaluation results are supported by corresponding evidence, thereby improving the interpretability and traceability of the evaluation results, and making it easier for textbook writers to make targeted modifications.

[0021] (3) By constructing a cross-stage problem chain continuity detection mechanism, this invention analyzes the problem connection relationship between different stages of the 5E textbook, and can discover situations such as problem target deviation, cognitive object transformation and problem solution chain interruption, thereby realizing the evaluation of the overall teaching logic quality of the textbook and improving the ability of the evaluation results to reflect the overall design quality of the textbook.

[0022] (4) After completing the analysis of the existing evaluation dimensions, the present invention further conducts supplementary defect detection, which can discover important problems that are not covered by the existing evaluation dimensions but may affect the teaching effect, the realization of stage functions or the reader's understanding, thereby improving the ability to discover textbook quality problems and reducing the risk of missed detection.

[0023] (5) This invention introduces the results of logical continuity detection between stages into the comprehensive scoring process, and uses a logical breakage penalty mechanism to quantitatively correct the interruption of the problem-solving chain, so that the final score not only reflects the quality of local content, but also reflects the completeness of the overall teaching process of the textbook, thereby improving the consistency between the comprehensive scoring results and the actual teaching quality.

[0024] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a structural block diagram provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0028] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0029] The following is combined with Figure 1 The present invention will be further described in detail below, describing a state-driven interpretable evaluation system for 5E teaching materials: In this embodiment, as Figure 1 As shown, a state-driven interpretable evaluation system for 5E teaching materials is provided, including: a teaching material evaluation data construction module 1, a text parsing module 2, a stage goal analysis module 3, a logic break detection module 4, an evidence extraction module 5, a defect detection module 6, a dimension scoring module 7, a result summary module 8, and an evaluation state object module 9.

[0030] The textbook evaluation data construction module is used to acquire textbooks, divide the content of textbooks according to chapters and stages, add attribute annotations, and construct evaluation input objects.

[0031] (1) Divide the content of the textbook according to chapters and stages to form the original textbooks for each stage. Each stage's original textbooks include multiple textbook content segments.

[0032] Original textbook text source: 5E model textbook. This can be a textbook written using the 5E teaching model; or a 5E structured textbook reconstructed from existing textbooks.

[0033] The acquired textbook content is divided into chapters and stored using chapters as the basic processing unit. Each chapter is further divided into five stages according to the five teaching stages of the 5E textbook teaching model: Excitation, Exploration, Enhancement, Execution, and Evaluation. The original textbook content fragments corresponding to each stage are organized and stored as the smallest analytical unit for subsequent evaluation processing.

[0034] (2) Add attribute annotations to the original textbook library and construct evaluation input objects.

[0035] Add attribute annotations to the original textbooks. These annotations include: stage labels, subject labels (e.g., Chinese language), and grade level labels (e.g., Grade 11). The annotation content is either manually added or automatically extracted from the textbook metadata.

[0036] For textbooks that already have a phased structure, their phase identifiers are read directly. For textbooks without explicitly marked phase information, the labelers divide the textbooks into phases according to the preset phase definitions and add phase tags to the corresponding chapters. The phase tags of the original textbooks for each phase are used to identify the 5E teaching phase to which the current textbook content segment belongs, and the phase text is used to store all textbook content segments corresponding to that phase.

[0037] The evaluation input includes subject tags, grade level tags, and excerpts of textbook content for each chapter and each stage, along with their stage tags.

[0038] Through the above process, the unified organization and input of textbook content, stage information, subject information, and grade level information are achieved, providing basic data support for subsequent text analysis, stage goal analysis, logical break detection, and dimensional scoring.

[0039] The evaluation status object module is used to construct a unified evaluation status object. The evaluation status object stores the evaluation input objects, all intermediate results generated during the textbook evaluation process, as well as the preset analysis objectives, stage objectives, evaluation dimensions, and scoring rules.

[0040] In this embodiment, after obtaining the evaluation input object of the teaching material, a unified evaluation state object is constructed to store the evaluation input object and all intermediate results generated during the evaluation process of the teaching material (generated in subsequent steps).

[0041] The evaluation status object serves as the data center for the entire evaluation process, providing a unified data read / write interface for subsequent text parsing, stage target analysis, logical break detection, evidence extraction, defect detection, dimensional scoring, and result summarization.

[0042] Unlike existing evaluation methods where each module reads and analyzes the original textbook text independently, this embodiment constructs an evaluation status object to achieve centralized management and sharing of evaluation information, avoiding repeated parsing of textbook content and improving the consistency and efficiency of the evaluation process.

[0043] (1) Structural design of evaluation status objects. Evaluation status objects adopt a hierarchical structure to organize evaluation data, mainly including the fact status layer and the evaluation status layer. The fact status layer records the content already identified by the system, including: core issues, knowledge points, activities, objectives, evidence, etc. The evaluation status layer records the system's judgment on the quality of teaching materials: degree of objective achievement, degree of logical continuity, defects, scoring status, etc.

[0044] (2) Initialization of the evaluation status object. After generating the evaluation input object, the subject label, grade level label, textbook content fragment, and stage label are read, and the evaluation status object is initialized. At this time, only the basic information is written, and the remaining analysis areas remain empty, waiting for subsequent modules to fill them in gradually.

[0045] (3) Evaluate the state update mechanism of the state object.

[0046] In the subsequent textbook evaluation process, each module uses the evaluation status object as the input and output carrier.

[0047] After the text parsing module completes the parsing, it writes the parsing result into the text parsing field; The phase target analysis module reads the text parsing field and writes the analysis results into the target analysis field; The logical break detection module reads the text parsing and target analysis fields and writes the detection results into the logical break field; The evidence extraction module reads text parsing fields, etc., and writes the extracted evidence set into the evidence pool field; The defect detection module reads the evidence pool fields, etc., and writes the defect detection results into the defect field; The dimension scoring module reads the evidence pool and the defect field, and writes the scoring results into the dimension scoring field; The results summary module reads all fields and generates the final comprehensive evaluation report of the teaching materials.

[0048] Through the above incremental update mechanism, the evaluation status objects gradually accumulate the analysis results generated by each module, thereby forming a complete information chain for textbook evaluation.

[0049] (4) Evaluate the state sharing mechanism of state objects.

[0050] The evaluation status object maintains a unique instance throughout the entire evaluation process. During execution, each module can read both the evaluation input object and the intermediate results already generated within the evaluation status object. The evaluation input object, serving as a unified data source, is consistently stored in the original content field. Modules can directly access this field when they need to obtain contextual information, supplement evidence, or verify results. For parsing and analysis results already completed by previous modules, the corresponding field of the evaluation status object is read first, without needing to execute the same analysis process again.

[0051] For example, when analyzing the achievement of stage goals, the stage goal analysis module can simultaneously read the original textbook text and the structured parsing results in the text parsing field; when generating scores, the dimension scoring module can read the original textbook text as the scoring basis, and directly utilize the goal analysis field and the evidence pool field, without having to conduct goal analysis again.

[0052] Through this sharing mechanism, the system retains the ability to access the textbook content fragments (original textbook text) of the evaluation input objects, while reusing the results of previous analysis, avoiding repeated parsing and reasoning, and improving the execution efficiency, consistency and interpretability of the evaluation process.

[0053] The evaluation status object records the entire analysis process and related intermediate results from textbook input to final scoring, providing complete data support for evaluation result traceability, defect location, and subsequent optimization.

[0054] The text parsing module is used to perform semantic analysis on the evaluation input objects according to the preset parsing objectives of each stage, identify key teaching elements, and index the text content of the evaluation input objects corresponding to the key teaching elements to obtain the parsing results.

[0055] For each stage's evaluation input object, the text parsing module reads the textbook content fragments and their stage tags from the evaluation status object, determines the parsing target based on the stage tags, performs semantic analysis and structured processing on the textbook content fragments in conjunction with the corresponding stage's parsing target, and writes the obtained structured parsing results into the text parsing field of the evaluation status object.

[0056] (1) Construction of analytical objectives for each stage.

[0057] Since different teaching stages undertake different teaching functions, the system first determines the types of teaching elements that need to be focused on based on the stage to which the textbook belongs, and then determines the analysis objectives.

[0058] In this embodiment, the analytical objectives for each stage of the 5E teaching model are as follows: The Excitation stage focuses on: scenario creation, problem posing, and the thinking process. The Exploration stage focuses on: the object of inquiry, the analytical path, the essential problem, and exploratory outputs. The Enhancement stage focuses on: the content of knowledge points, the source of knowledge points, and the learning structure and organization. The Execution stage focuses on: the practical steps, the practical methods, and the practical results. The Evaluation stage focuses on: summarizing the problem-solving process, reflective activities, and cognitive transfer.

[0059] (2) Structured analysis of textbook content.

[0060] The system performs semantic analysis on textbook content fragments, identifies key teaching elements in the textbook, and creates an index for the textbook content fragments corresponding to the key teaching elements.

[0061] In one implementation, the text parsing module completes the task of identifying teaching elements by calling a large language model. The system inputs the textbook content fragments of the current stage (Excitation, Exploration, Enhancement, Execution, or Evaluation stage) and the corresponding parsing target into the large language model of the textbook parsing role. The large language model then identifies the corresponding content (key teaching elements) in the textbook content fragments according to the stage target. For the key teaching elements identified in each stage, corresponding structured records are generated.

[0062] For example, the recording of key teaching elements in the excitation stage may include the following information: the context or phenomenon in which the text is initially presented, the observation, thinking or analysis process that takes place in the middle of the text, the core question that the text finally addresses, and the problems that exist in the text.

[0063] The text parsing module extracts the text content corresponding to key teaching elements from fragments of textbook content and creates an index for them. Parsing results are generated based on the key teaching elements and their corresponding textbook content fragment indexes.

[0064] (3) The parsing results are written into the evaluation status object.

[0065] After text parsing is complete, the text parsing module writes the structured parsing results into the text parsing field of the evaluation status object. Once written, the evaluation status object contains a structured representation of the textbook content.

[0066] The target analysis module, logic break detection module, evidence extraction module, and defect detection module in subsequent stages can all directly read the text parsing field to obtain the textbook structure information and read the corresponding textbook content fragments through this index, without having to parse the original textbook text (all textbook content fragments) for this stage again.

[0067] After the text parsing module generates the parsing results but before writing them to the evaluation status object, a consistency check can be performed on the parsing results to ensure their integrity. The check includes: whether the key teaching elements of the current stage have been identified, whether any fields have been omitted, and whether any structured results are empty. If the consistency check fails, the text parsing module re-executes the parsing process and supplements the parsing results.

[0068] The stage goal analysis module is used to perform teaching objective matching analysis on the analysis results and evaluation input objects based on preset stage goals, and obtain stage goal analysis results.

[0069] After text parsing is complete, the system enters the stage goal analysis phase. The stage goal analysis module is used to determine whether the teaching material content meets the teaching objectives that should be achieved in the current teaching stage, and writes the analysis results into the goal analysis field of the evaluation status object.

[0070] Unlike methods that directly generate evaluation results based on textbook content, this invention first constructs the stage goal achievement status, and then conducts subsequent logical break detection, defect detection and dimensional scoring based on the goal achievement status, so that the subsequent evaluation is based on clear stage goals.

[0071] (1) Construction of the stage target library.

[0072] The system pre-establishes a stage goal database, which stores the stage goals corresponding to different teaching stages. Each stage includes one or more goal items, and each goal item corresponds to the teaching behavior or learning activity that should be reflected in the current stage.

[0073] For example, the goals of the Exploration phase include: starting from the specific questions raised in the Excitation, abstracting the essential questions behind them; breaking down and analyzing the questions based on basic principles, key concepts, or preliminary logic; forming a deeper understanding of the problem structure and clarifying the direction for subsequent learning, verification, or solving; and maintaining the "exploratory" nature, that is, focusing on discovering the essence of the problem and clarifying the research direction, rather than prematurely entering into knowledge explanation, method teaching, or specific operations.

[0074] (2) Target matching analysis.

[0075] The stage goal analysis module reads the evaluation input objects and parsing results from the evaluation status object, and determines the set of stage goals that should be loaded at the moment based on the subject label, grade level label, and stage label.

[0076] The stage goal analysis module performs a goal-by-goal analysis of the analysis results based on the current stage goals. For each goal, the module comprehensively utilizes key teaching elements and their corresponding textbook content segments, as well as the stage goals, to determine whether the textbook reflects the requirements of the corresponding stage goals. The textbook content segments are extracted from the evaluation input objects using the index of the analysis results.

[0077] In one implementation, the stage goal analysis module performs goal matching analysis through a large language model. The stage goal analysis module inputs the stage goals, analysis results, and evaluation input objects into the large language model, which outputs the following for each goal item: goal name, achievement status (achieved or missing), supporting evidence, and reason for achievement or reason for missing. After completing the analysis of all goal items, the large language model generates the stage goal analysis results for the current textbook.

[0078] (3) The results of the phase target analysis are written into the evaluation status object.

[0079] After the target matching analysis is completed, the stage target analysis module writes the stage target analysis results into the target analysis field of the evaluation status object. Through this structured representation, an explicit correspondence is established between the teaching material content and the stage targets.

[0080] In addition, similar to the consistency check of the parsing results, this embodiment can perform consistency verification on the stage target analysis results to ensure the integrity of the analysis results. The verification includes: whether all target items in the current stage are covered; whether there are any targets that have not been analyzed; and whether there are any situations where the target status clearly contradicts the textbook content. When incomplete or abnormal stage target analysis results are detected, the target matching analysis process is re-executed, and the results are written to the target analysis field in the evaluation status object. Through the above processing, the system forms a target achievement status table between the textbook content and the stage targets, providing a unified analytical basis for subsequent logical break detection, defect detection, and dimensional scoring.

[0081] The logic break detection module is used to analyze the continuity of the problem chain between stages based on the parsing results and the stage target analysis results, and to obtain the logic break results.

[0082] After completing the phase goal analysis, the system enters the logic break detection phase. The logic break detection module is used to determine whether the current phase continues the core problem, core assumption, or core cognitive object proposed in the previous phase in terms of problem logic, and writes the detection result (logic break result) into the logic break field of the evaluation status object.

[0083] Unlike the defect detection module, which focuses on the quality of teaching materials, this module does not evaluate the quality of the material's expression, activity design, or the achievement of dimensions. Instead, it focuses solely on whether the teaching materials maintain a continuous problem-solving chain between stages. In the 5E teaching model, the stages are not independent but rather a progressive problem-solving process centered around the same core issue. Specifically, around the same core issue, the Excitation stage is responsible for raising the question, forming the core question and research object for subsequent learning; the Exploration stage is responsible for analyzing the essence of the problem and proposing solutions and preliminary plans; the Enhancement stage is responsible for supplementing the solution with the necessary knowledge, methods, and tools, forming the necessary implementation capabilities; the Execution stage is responsible for implementing the solution using the knowledge and methods developed in the previous stage, completing the problem-solving; and the Evaluation stage is responsible for evaluating the implementation results, analyzing shortcomings, formulating improvement suggestions, and completing the closed loop of the entire problem-solving process. Therefore, the logical break detection module focuses on analyzing the continuity of the problem between the current stage and the previous stage.

[0084] (1) Construction of input for problem chain analysis.

[0085] The logic break detection module reads the following information from the evaluation status object: parsing results, stage target analysis results, and current stage information. If necessary, it will also read the evaluation input objects for the current stage and the previous stage.

[0086] Among them, the analysis results of the current stage and the previous stage serve as the main analytical basis for logical break detection, and are used to obtain the key teaching elements (core questions, cognitive objects, inquiry objectives, knowledge topics and teaching tasks, etc.) that have been identified in the previous stage and the current stage.

[0087] The results of the phase goal analysis are used to obtain the achievement status of the phase goals and provide auxiliary judgment basis for the continuity analysis of the problem chain.

[0088] The current stage information is used to determine the current stage's position in the 5E teaching process, thereby selecting the corresponding problem chain analysis strategy.

[0089] The evaluation inputs in the current and previous stages are not the main objects of analysis, but rather serve as semantic supplementary information to help the model understand the context and to verify the consistency of the structured analysis results.

[0090] (2) Problem chain continuity analysis.

[0091] The logic break detection module analyzes the continuity of the problem chain between the current stage and the previous stage based on the analysis results of the current stage and the previous stage. It mainly analyzes the connection between stages and detects whether the current stage effectively inherits the key deliverables formed in the previous stage, including core issues, solutions, knowledge requirements, knowledge results or implementation results.

[0092] In one implementation, the logic break detection module performs a problem chain continuity analysis task through the large language model of the logic break detection role.

[0093] The logic break detection module inputs the following into the large language model: the parsing results of the previous stage, the parsing results of the current stage, the analysis results of the current stage's stage objectives, the current stage information, and the problem chain continuity determination rules.

[0094] This large language model focuses on analyzing the following: Consistency analysis of core issues: Determine whether the current stage still revolves around the core issues raised in the previous stage.

[0095] Cognitive object consistency analysis: Determine whether the cognitive object being focused on in the current stage is consistent with that in the previous stage.

[0096] Problem progression relationship analysis: Determine whether to continue addressing the problems raised in the previous stage at the current stage.

[0097] Reasoning sequence analysis: Determine whether the current stage should utilize the conclusions, assumptions, or cognitive results formed in the previous stage to continue subsequent activities.

[0098] (3) Determination of logical breaks.

[0099] The logic break detection module determines the logic break result based on the problem chain continuity analysis results output by the large language model of the logic break detection role.

[0100] In this embodiment, the logic break results are divided into the following four levels: No logical breaks: The current stage clearly revolves around the core issues, core assumptions, or cognitive objects raised in the previous stage, the problem chain remains continuous, and each stage serves the same problem-solving goal.

[0101] Mild logical breaks: The current stage still revolves around the same problem object, but the connection to the key assumptions, key reasoning nodes or existing conclusions in the previous stage is not clear enough; the problem chain still exists, but readers need to fill in some logical connections themselves.

[0102] Moderate logical break: The problem objective of the current stage has shifted; although it can still be related to the previous stage, the content of the teaching materials no longer focuses on solving the previous problems.

[0103] Severe logical break: The current stage completely deviates from the problem objectives of the previous stage, introducing new problem objects, knowledge topics, or teaching tasks, resulting in the interruption of the problem-solving chain.

[0104] When the logical break level cannot be accurately classified based on the results of the problem chain continuity analysis, the textbook content fragments of the evaluation input objects of the current stage and the previous stage are read based on the index of the parsing results, and these fragments are also input into the large language model of the logical break detection role, and the problem chain continuity analysis task is executed again.

[0105] (4) The logical break result is written into the evaluation status object.

[0106] After completing the logic break analysis, the logic break detection module writes the logic break results into the logic break field of the evaluation status object in the form of a teaching logic relationship diagram.

[0107] The teaching logic diagram is used to describe the logical relationships between different chapters, stages, and teaching activities in the textbook. The nodes in the diagram represent key teaching elements such as core questions, cognitive objects, and inquiry objectives. The edges in the diagram represent the connections between nodes, including sequential, supporting, causal, or verification relationships.

[0108] The logic break detection module analyzes the connection between the corresponding logical nodes of the previous stage and the current stage to identify whether there is a logic break, and fills the logic break result into the corresponding edge.

[0109] The content written into the teaching logic relationship diagram of the evaluation state object by the logic break detection module includes: node information, logic break level (such as no logic break, moderate logic break) and cause explanation (the problem chain continuity analysis results output by the large language model).

[0110] The evidence extraction module is used to extract evidence from the evaluation input objects based on preset evaluation dimensions, analysis results, and stage target analysis results, construct an explicit evidence chain, and obtain an evidence pool.

[0111] After completing the logic break detection, the system enters the evidence extraction stage. The evidence extraction module is used to extract key evidence from the textbook content that can support subsequent defect detection and dimension scoring, establish the correspondence between the evaluation conclusion and the original text of the textbook, and finally write the evidence set (evidence pool) into the evidence pool field of the evaluation status object.

[0112] Unlike methods that directly rely on models to generate scoring results, this invention constructs an explicit chain of evidence before scoring, enabling subsequent defect detection and scoring processes to be traced back to the corresponding textbook content, thereby improving the interpretability and consistency of the evaluation results.

[0113] (1) Construction of evidence extraction input.

[0114] The evidence extraction module reads the following information from the evaluation status object: parsing results, stage target analysis results, evaluation input objects, and preset evaluation dimensions.

[0115] The analysis results serve as the primary basis for analysis, used to extract the structured key teaching elements from the textbook.

[0116] The results of the phase goal analysis are used to obtain information on the achievement of phase goals.

[0117] The current stage of evaluation input is used to locate specific evidence content and verify the extraction results.

[0118] Evaluation dimensions are used to determine evaluation objectives, evaluation indicators, and the scope of evidence extraction, thereby guiding the evidence extraction module to identify supporting evidence related to the evaluation dimension from the textbook content fragments of the evaluation input object.

[0119] For example, the scoring dimensions for the Excitation stage include: the realism and immersion of the context, the clarity of the problem presentation, the solvability of the problem, and its suitability to the student's cognitive state.

[0120] It should be noted that evidence extraction is carried out across all evaluation dimensions of all textbook content at all stages, but the evidence extraction module constructs evidence extraction inputs for each evaluation dimension of each stage in turn.

[0121] (2) Evidence extraction and correlation analysis The evidence extraction module identifies key teaching elements from the analysis results based on the current evaluation dimension. Combining this with the achievement status of these key teaching elements from the stage goal analysis results, it extracts evidence supporting the evaluation judgment from textbook content fragments. Where the achievement status represents a missing key teaching element, and no corresponding textbook content fragment exists, evidence extraction is unnecessary.

[0122] In one implementation, the evidence extraction module performs the evidence extraction task through a large language model of the evidence extraction role. The current stage's parsing results, stage target analysis results, evaluation input objects, and the current evaluation dimension are simultaneously input into the large language model, and the model outputs the evidence content of the current stage related to the current evaluation dimension.

[0123] (3) Construction of the evidence pool.

[0124] After completing the evidence extraction for all stages and all evaluation dimensions, the evidence extraction module summarizes all evidence content, establishes the correspondence between evidence content and evaluation dimensions, merges duplicate evidence, constructs an evidence pool, and realizes unified organization and management of the evidence content corresponding to each evaluation dimension.

[0125] The evidence pool is used to centrally store various types of extracted evidence and establish the correlation between evaluation dimensions, evaluation indicators, and evidence content. For the same evaluation dimension, the evidence extraction module collects relevant evidence content from different text locations, different teaching activities, and different knowledge stages to form an evidence set for the corresponding dimension. For evidence that supports multiple evaluation dimensions, cross-dimensional correlations are established to avoid duplicate extraction and duplicate judgment in subsequent analysis.

[0126] The constructed evidence pool not only stores the specific evidence content but also the mapping relationship between the evidence and the evaluation dimensions. This allows subsequent defect detection and dimension scoring modules to directly access the evidence set for the corresponding dimension for analysis without having to re-search the original textbook text. Through the evidence pool mechanism, unified management, shared reuse, and traceable analysis of evaluation evidence are achieved, improving the efficiency, consistency, and interpretability of subsequent evaluation processes.

[0127] (4) Verification of the integrity of the evidence pool The evidence extraction module performs a completeness check on the extraction results. The check includes: whether the evidence content covers the key teaching elements corresponding to the current evaluation dimension; whether there is duplicate evidence; and whether the evidence content contains content irrelevant to the evaluation dimension. If the completeness check fails, the evidence extraction module re-executes the evidence extraction process.

[0128] (5) The evidence results are written into the evaluation status object. The evidence extraction module writes the completed evidence pool into the evidence pool field of the evaluation status object.

[0129] The subsequent defect detection module determines whether the teaching materials have problems such as missing key evidence, insufficient evidence, or mismatch between evidence and evaluation dimensions by reading the fields in the evidence pool; the dimension scoring module generates scoring results and scoring basis based on the evidence content in the evidence pool.

[0130] By extracting evidence, the system establishes an explicit chain of evidence between the evaluation dimensions and the textbook content, enabling the evaluation conclusions to be traceable and interpretable.

[0131] The defect detection module is used to determine the evaluation coverage based on the analysis results, stage target analysis results, evaluation input objects, logic break results, and evidence pool. Outside the evaluation coverage, it performs supplementary defect analysis on the content of the teaching materials and generates defect descriptions.

[0132] After evidence extraction is completed, the system enters the defect detection phase. The defect detection module is used to identify important quality issues that are not fully covered by existing evaluation dimensions, and to supplement the analysis of potential defects in the teaching materials. Finally, the detection results (defect descriptions) are written into the defect field of the evaluation status object.

[0133] Unlike the aforementioned stage target analysis, logical break detection, and dimensional evaluation, the defect detection module does not repeat the existing evaluation rules. Instead, it takes a holistic approach to the quality of the teaching materials and conducts supplementary detections on issues that may be missed by the existing evaluation system.

[0134] (1) Construction of defect detection input.

[0135] The defect detection module reads the following information from the evaluation status object: parsing results, stage target analysis results, evaluation input objects, logic break results, evidence pool, and the current evaluation dimension.

[0136] Similar to evidence extraction, defect detection is also carried out across all evaluation dimensions of all stages of the textbook content. The defect detection module constructs defect detection inputs for each evaluation dimension of each stage in turn.

[0137] (2) Evaluation coverage analysis and supplementary defect discovery.

[0138] Based on the defect detection input, the defect detection module first analyzes the scope of quality issues already covered by the current evaluation system to obtain the evaluation coverage. This coverage includes: completed evaluation dimensions, identified logical breakage issues, achieved target results, and extracted supporting evidence. By constructing the evaluation coverage, the defect detection module can identify the boundaries of content already analyzed during the current evaluation process. The evaluation coverage is used to subsequently determine whether a particular issue has been adequately covered by the existing dimensions.

[0139] Beyond the scope of the evaluation, the defect detection module performs supplementary analysis on the textbook content, generates defect descriptions, and performs additional defect analysis.

[0140] In one implementation, the defect detection module performs a supplementary defect discovery task through a large language model with a defect supplement detection role. The parsing results, stage target analysis results, evaluation input objects, logical break results, evidence pool, and the current evaluation dimension are input into the large language model. The large language model calculates defects outside the evaluation coverage. The defect must simultaneously meet the following conditions: the defect is a defect existing in the teaching materials other than the existing evaluation dimensions; the defect affects the teaching effect; the defect affects the implementation of stage functions; and the defect affects the reader's understanding.

[0141] For the deficiencies that are finally confirmed, the large language model generates a structured description of the deficiencies, including: the location of the deficiencies, the content of the deficiencies, the specific manifestations, and the impact analysis.

[0142] (3) The defect results are written into the evaluation status object.

[0143] The defect detection module writes defect descriptions into the defect field of the evaluation status object. By introducing a defect correction mechanism, the system can not only evaluate the performance of teaching materials on existing evaluation dimensions, but also incorporate quality issues that are not fully covered by evaluation dimensions or that are difficult for evaluation rules to directly reflect into the scoring process, thereby reducing scoring bias and improving the completeness, accuracy, and robustness of teaching material evaluation results.

[0144] The dimension scoring module is used to score based on the preset scoring rules corresponding to the evaluation dimensions, and to generate dimension scoring results according to the evidence pool, stage target analysis results, logical break results, and defect descriptions.

[0145] After defect detection is completed, the system enters the dimension scoring generation stage. The dimension scoring module is used to classify and score the performance of the teaching materials in each evaluation dimension according to the pre-established scoring rules, and write the scoring results into the dimension scoring field of the evaluation status object.

[0146] Unlike evaluation methods that directly generate scores based on textbook content, this invention first completes target analysis, logical break detection, evidence extraction, and defect detection, and then maps scores based on the above analysis results, thereby making the scoring process clear and interpretable.

[0147] (1) Construction of scoring input.

[0148] The dimension scoring module reads the following information from the evaluation status object: evidence pool, stage target analysis results, logic break results, defect description, evaluation dimensions and corresponding scoring rules.

[0149] (2) Construction of the scoring rule base.

[0150] The system pre-establishes a scoring rule library to store the scoring rules and grading rules corresponding to all evaluation dimensions. This embodiment uniformly adopts the 0-5 point grading rule.

[0151] This embodiment also designs scoring principles and constraints for evaluation dimensions, including: Dimensional Independence Principle: Each evaluation dimension is scored independently, and the scores of other evaluation dimensions should not affect the score of the current dimension.

[0152] Core Function Priority Principle: The focus of the scoring is not on whether the text is beautifully expressed, but on whether the teaching function corresponding to the current dimension is effectively implemented.

[0153] Understanding and Usage Constraints: If there are problems affecting understanding, use, or teaching implementation in the current evaluation dimension, the score for that dimension shall not exceed 3 points.

[0154] Key sub-point constraints: If any key sub-point is not satisfied, the highest score for the current evaluation dimension shall not exceed 2 points; if all key sub-points are satisfied, but there are missing non-key sub-points, the score shall not exceed 3 points; only when all key sub-points and non-key sub-points are satisfied can the score range of 4 points or above be entered.

[0155] To enhance the focus on core functionalities during the evaluation process, this embodiment predefines key and non-key sub-points within each evaluation dimension. Key sub-points are manually annotated by domain experts based on the evaluation objectives and are written into the configuration file for the corresponding evaluation dimension when building the evaluation rule base. Each sub-point includes attributes such as a unique identifier, its associated dimension, and a key marker.

[0156] Non-critical sub-points refer to evaluation content within an evaluation dimension that, apart from critical sub-points, can supplement, strengthen, or improve the teaching function corresponding to that dimension, but their absence usually does not directly render the core function of that dimension invalid. Non-critical sub-points should be determined based on the evaluation objectives and teaching functions of the current dimension, and should have a clear semantic and functional relationship with the dimension. They should not introduce general quality requirements that lack a direct connection to the current dimension, such as language norms, formatting, or general content completeness.

[0157] For example, sub-points of the scoring dimensions for the realism and immersion of the context construction in the excitation stage include: Key sub-point: Whether the first-person narrative is used to present a real-life situation that is closely related to the learning task, close to students' experience, or understandable; Non-critical sub-points: Does it have storytelling or role-playing elements to enhance immersion and emotional engagement? Non-critical sub-point: Whether the context can generate initial problem awareness, rather than just providing background information.

[0158] (3) Evaluation dimension achievement status analysis and score generation.

[0159] For each evaluation dimension, the dimension scoring module obtains the corresponding scoring rules and determines the key sub-points and non-key sub-points; it matches the stage target analysis results with the key sub-points and non-key sub-points one by one; and then, combined with the logical break results and defect descriptions, it makes a judgment on the achievement of the evaluation dimension and obtains the achievement status of the evaluation dimension.

[0160] The evaluation criteria for achieving the following dimensions mainly assess the following: whether the phase goal analysis results meet the key sub-points; whether the phase goal analysis results meet the non-key sub-points; whether there are any issues affecting understanding and use based on the phase goal analysis results; whether there are any logical breakage-related issues based on the logical breakage results; and whether there are any supplementary defects based on the defect description.

[0161] The dimension scoring module performs tiered matching on the achievement status of the evaluation dimensions and automatically generates dimension scores based on the matching results.

[0162] The tier matching rule in this embodiment is as follows: 0 points: The core functionality of the current dimension is completely absent, or there is obvious reverse behavior; 1 point: Only a small amount of surface-level content exists; the core functionality of the dimension is not implemented. 2 points: Some content is correct, but at least one key sub-point is missing; 3 points: All key points are satisfied, but there are missing non-key points or problems that affect understanding and use; 4 points: Both key and non-key points are satisfied, the overall logic is clear, and there are only a few minor flaws; 5 points: All requirements of the current dimension are met, and there are no reasonable suggestions for modification.

[0163] (4) Generation and writing of scoring reasons.

[0164] After the scoring is completed, the dimensional scoring module, in conjunction with the evidence pool, generates scoring criteria for each evaluation dimension. Each conclusion in the scoring criteria must be associated with corresponding evidence to ensure the traceability of the scoring process. The dimensional scoring results include the dimensional score and its corresponding scoring criteria.

[0165] Finally, the dimension scoring module writes the dimension scoring results into the dimension scoring field of the evaluation status object.

[0166] Through the dimensional scoring module, the system maps the evidence pool formed in the previous stage, the stage target analysis results, the logical break results, and the defect descriptions into standardized scoring results, realizing the automatic conversion from textbook analysis to quantitative evaluation.

[0167] The results summary module is used to generate a comprehensive evaluation report of the teaching materials based on the dimensional scoring results, logical break results, and defect descriptions.

[0168] After scoring each evaluation dimension, the system enters the stage of summarizing results and generating a comprehensive score.

[0169] The results summary module is used to perform weighted calculations on the dimensional scores of each evaluation dimension, and combine the logical break results and defect descriptions to generate the final evaluation score of the textbook, ultimately forming a comprehensive evaluation report of the textbook.

[0170] Unlike evaluation methods that simply average the scores of each dimension, this embodiment takes into account the characteristics of the 5E textbook, considering both the quality within each stage and the logical continuity between stages, thereby ensuring that the final score reflects the completeness of the overall problem-solving chain of the textbook.

[0171] (1) Reading the scoring results.

[0172] The results summary module reads the following information from the evaluation status object: dimensional scoring results, logical break results, defect descriptions, and preset weights.

[0173] (2) Weighted calculation of scores.

[0174] The results summary module calculates the stage score for the current stage based on the dimensional scores in the dimensional scoring results, according to preset weights.

[0175] The formula for calculating the score for the corresponding stage is: in, For the stage score, The score for the i-th evaluation dimension of the stage. For the i-th weight of the stage, This represents the total number of evaluation dimensions.

[0176] (3) Defect correction score.

[0177] Since some textbook quality issues may not be fully reflected by a single evaluation dimension, the system further adjusts the dimension weighted score by incorporating defect descriptions to generate a defect-corrected score. The defect impact coefficient (derived from expert annotations in the defect database) is determined based on the degree of impact of the defect descriptions at the current stage, and the stage score is then adjusted to obtain the defect-corrected score.

[0178] in, This is the score after defect correction. This is the defect impact coefficient; the higher the coefficient, the smaller the impact.

[0179] (4) Logical break correction score.

[0180] Since the 5E textbook involves a continuous problem-solving process, evaluating only the internal quality of each stage is insufficient to reflect the overall quality of the textbook. The results summary module further generates a logical continuity penalty coefficient based on the logical breakage results.

[0181] The 5E teaching model exhibits a clear progressive nature, with the outcomes of each stage serving as the foundation for subsequent activities. For instance, the core questions raised in the Excitation stage should be explored in the Exploration stage, the findings from the Exploration stage should be used to construct knowledge in the Enhancement stage, and subsequent stages should further facilitate knowledge transfer and reflective evaluation.

[0182] Therefore, the quality of teaching materials in the 5E teaching model depends not only on the completion of each evaluation dimension, but also on whether a complete, continuous, and traceable logical chain of teaching is formed between each teaching stage. If there are logical breaks between stages, even if the quality within each stage is high, it will affect the integrity of the entire 5E teaching process. Therefore, this invention introduces a logical continuity penalty coefficient on the basis of dimension scoring to make an overall correction to the comprehensive score of the teaching materials.

[0183] The logic continuity penalty coefficient follows these design principles: Global Correction Principle: Logical breaks are cross-stage quality issues, and their impact typically involves multiple evaluation dimensions, not just a single one. Therefore, instead of deducting points for logical breaks as a single evaluation dimension, a global penalty coefficient is used to correct the overall score, ensuring that the score reflects the completeness of the teaching process as a whole.

[0184] The principle of phased progression: The 5E teaching model has a strict phase dependency relationship, with each subsequent phase building upon the previous one. Therefore, logical breaks occurring in early phases will continue to affect the subsequent teaching process, while new logical breaks in later phases will further reduce the overall quality of the teaching materials. This invention adopts a phased progressive penalty mechanism, allowing the impact of logical breaks to accumulate continuously throughout the evaluation process.

[0185] The principle of graded impact: Different logical breaks have varying degrees of impact on the quality of teaching materials. This invention employs different penalty strategies for different levels of logical breaks based on the degree to which the logical chain of teaching is disrupted.

[0186] The initial test penalty coefficient design: The initial test occurs during the first logical continuity analysis between the current stage and the previous stage, and its purpose is to establish the initial credibility of the overall teaching logic of the textbook. Based on the progressive characteristics of the 5E teaching model, this invention adopts a larger penalty range in the initial test, reflecting the credibility of the initial teaching logic of the textbook by directly setting a logical continuity penalty coefficient. The higher the degree of logical break, the lower the possibility that subsequent teaching activities can be established on a complete logical chain, and therefore a greater overall correction is given accordingly.

[0187] The initial detection logic continuity penalty coefficient in this embodiment is shown in Table 1.

[0188] Table 1

[0189] The initial assessment uses fixed assignments rather than progressive corrections because this process establishes an initial evaluation benchmark for the overall teaching logic of the textbook. Only after determining whether the textbook possesses a complete logical foundation can subsequent stages of logical continuity analysis and scoring corrections be truly meaningful.

[0190] The design of penalty coefficients for subsequent stages where logical breaks occur in consecutive stages is as follows: For the stage where a logical break between stages is detected for the first time, the logical continuity penalty coefficient is determined in the manner described above. For subsequent stages where logical breaks occur consecutively, the logical continuity penalty coefficient is not re-determined based on the existing evaluation status. Instead, the value of the logical continuity penalty coefficient is progressively adjusted to reduce its value. For logical breaks detected in subsequent stages, the system selects an appropriate correction ratio based on the logical break level and cumulatively updates the current penalty coefficient, ensuring that logical breaks from multiple stages can collectively affect the comprehensive evaluation result of the teaching materials.

[0191] Among them, mild logical breaks only have a local impact on the current teaching segment, so a small correction is adopted, such as 0.75×0.95; moderate logical breaks indicate that there is a significant lack of connection between stages, which further affects the overall teaching process of the textbook, so a larger correction ratio is adopted, such as 0.5×0.90; severe logical breaks indicate that there is no effective teaching connection between the current stage and the previous stage, so the logical continuity penalty coefficient is still 0, in order to fully reflect the cumulative impact of severe logical breaks on the overall quality of the textbook.

[0192] Using a progressive update method instead of reassigning values ​​ensures that the logical quality assessment formed in the early stages is retained and new logical influences are continuously accumulated as the textbook evaluation process progresses. This is more in line with the progressive and interdependent characteristics of the 5E teaching model.

[0193] The results summary module uses a logical continuity penalty coefficient to correct the score after defect correction, and obtains a comprehensive score.

[0194] The calculation formula is: in, The PenaltyCoefficient is the logical continuity penalty coefficient for the overall score.

[0195] (5) Formation and output of comprehensive evaluation report of teaching materials.

[0196] The results summary module generates a comprehensive evaluation report for the teaching materials. This report includes: overall score, dimensional scoring results, logical breakage results, and defect descriptions. It may also include stage scores for each phase and scores after defect corrections.

[0197] Through the results summary module, the system can not only evaluate the performance of the teaching materials in each evaluation dimension, but also evaluate the continuity and integrity of the teaching materials in the entire 5E problem-solving chain, thereby generating a comprehensive evaluation report that is more in line with the principles of instructional design.

[0198] The system works as follows: Users upload textbooks to the textbook evaluation data construction module. The textbook evaluation data construction module divides the content of the textbooks according to chapters and stages, adds attribute annotations, and constructs evaluation input objects; then it transmits the evaluation input objects to the evaluation status object module. The evaluation status object module initializes the evaluation status object based on the evaluation input object, stores the evaluation input object in the evaluation status object, and retrieves the corresponding preset parsing objectives, stage objectives, evaluation dimensions and scoring rules based on the stage tags, subject tags and grade level tags in the evaluation status object, which are also stored in the evaluation status object. The text parsing module reads the parsing target and the evaluation input object from the evaluation status object, performs semantic analysis on the evaluation input object according to the preset parsing target of each stage, identifies key teaching elements, and establishes an index for the text content of the evaluation input object corresponding to the key teaching elements to obtain the parsing result; the parsing result is transmitted to the evaluation status object module and stored in the evaluation status object; The stage goal analysis module reads the stage goals from the evaluation status object, performs teaching goal matching analysis on the parsing results and evaluation input objects, and obtains the stage goal analysis results; the stage goal analysis results are then transmitted to the evaluation status object module and stored in the evaluation status object. The logic break detection module reads the parsing results and stage target analysis results from the evaluation status object, analyzes the continuity of the problem chain between stages, and obtains the logic break result; the logic break result is then transmitted to the evaluation status object module and stored in the evaluation status object. The evidence extraction module reads the evaluation dimensions, analysis results, stage target analysis results, and evaluation input objects from the evaluation status object. Based on the analysis results and stage target analysis results, it extracts evidence from the evaluation input objects, constructs an explicit evidence chain, and obtains the evidence pool. The evidence pool is then transmitted to the evaluation status object module and stored in the evaluation status object. The defect detection module reads the parsing results, stage target analysis results, evaluation input objects, logic break results, and evidence pool from the evaluation status object to determine the evaluation coverage. Outside the evaluation coverage, it performs supplementary defect analysis on the content of the teaching materials and generates defect descriptions. The defect descriptions are then transmitted to the evaluation status object module and stored in the evaluation status object. The dimensional scoring module reads the scoring rules, evidence pool, stage target analysis results, logical break results, and defect descriptions from the evaluation status object, performs scoring, and generates dimensional scoring results; the dimensional scoring results are then transmitted to the evaluation status object module and stored in the evaluation status object. The results summary module reads the dimension scoring results, logical break results, and defect descriptions from the evaluation status object and generates a comprehensive evaluation report for the teaching materials.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A state-driven interpretable evaluation system for 5E teaching materials, characterized in that, include: The textbook evaluation data construction module is used to acquire textbooks, divide the content of textbooks according to chapters and stages, add attribute annotations, and construct evaluation input objects; The text parsing module is used to perform semantic analysis on the evaluation input objects according to the preset parsing objectives of each stage, identify key teaching elements, and index the text content of the evaluation input objects corresponding to the key teaching elements to obtain the parsing results. The stage goal analysis module is used to perform teaching goal matching analysis on the analysis results and evaluation input objects based on preset stage goals, and obtain the stage goal analysis results. The logic break detection module is used to analyze the continuity of the problem chain between stages based on the parsing results and the stage target analysis results, and to obtain the logic break results. The logic break detection module analyzes the continuity of the problem chain between the previous stage and the current stage to obtain the logic break result; Among them, the problem chain continuity analysis examines the connection between analysis stages and detects whether the current stage effectively inherits the key results formed in the previous stage; it includes core problem consistency analysis, cognitive object consistency analysis, problem progression relationship analysis, and reasoning connection relationship analysis. The evidence extraction module is used to extract evidence from the evaluation input objects based on the preset evaluation dimensions, the analysis results and the stage target analysis results, construct an explicit evidence chain, and obtain an evidence pool. The defect detection module is used to determine the evaluation coverage based on the analysis results, stage target analysis results, evaluation input objects, logic break results, and evidence pool. Outside the evaluation coverage, it performs supplementary defect analysis on the content of the teaching materials and generates defect descriptions. The dimension scoring module is used to score based on the preset scoring rules corresponding to the evaluation dimensions, according to the evidence pool, stage target analysis results, logical break results, and defect descriptions, and generate dimension scoring results. The results summary module is used to generate a comprehensive evaluation report of the teaching materials based on the dimensional scoring results, logical break results, and defect descriptions.

2. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 1, characterized in that, The textbook evaluation data construction module divides the textbook content by chapter, and further divides the content of each chapter into stages according to the five teaching stages of the 5E textbook, resulting in different textbook content fragments for each stage; Add attribute labels to obtain the evaluation input object; the attribute labels include: stage label, subject label, and grade level label.

3. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 1, characterized in that, For each stage of the evaluation input, the text parsing module determines the parsing target based on the stage label of the evaluation input; combined with the parsing target, it performs semantic analysis on the textbook content fragments of the evaluation input to identify key teaching elements; it establishes an index for the textbook content fragments corresponding to the key teaching elements; and it generates the parsing results based on the key teaching elements and their corresponding textbook content fragment indexes.

4. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 1, characterized in that, Each stage objective includes one or more objective items, which correspond to the teaching behaviors or learning activities of the current stage; Input the stage goals, analysis results, and evaluation inputs into the large language model. For each goal item, the large language model outputs: goal name, achievement status, supporting basis, and reasons for achievement or lack thereof.

5. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 1, characterized in that, The evidence extraction module locates the corresponding key teaching elements from the analysis results based on the evaluation dimensions, and extracts evidence content from the textbook content fragments by combining the achievement status of the key teaching elements in the stage goal analysis results. By summarizing all evidence content across all stages and evaluation dimensions, establishing the correspondence between evidence content and evaluation dimensions, an evidence pool is constructed.

6. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 1, characterized in that, For each evaluation dimension, the dimension scoring module matches the stage target analysis results with key sub-points and non-key sub-points point by point; then, combined with the logical break results and defect descriptions, it makes a judgment on the achievement of the evaluation dimension and obtains the achievement status of the evaluation dimension; it performs graded matching on the achievement status of the evaluation dimension to generate a dimension score and obtain the dimension scoring result, which includes the dimension score and the corresponding scoring basis. Among them, the key sub-point focuses on whether the teaching functions corresponding to the current dimension are effectively implemented.

7. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 1, characterized in that, The results summary module calculates the stage score for each stage based on the dimensional scoring results and preset weights, and corrects the stage scores based on the logical break results and defect descriptions to obtain the comprehensive score for each stage. Based on the comprehensive score, dimensional scoring results, logical break results, and defect descriptions, a comprehensive evaluation report of the teaching materials is generated.

8. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 7, characterized in that, The logic breakage detection module classifies logic breakage levels based on the logic breakage results. The results summary module obtains the logic continuity penalty coefficient based on the logic break level and corrects the stage score; for the stage where the logic break is detected for the first time, the corresponding logic continuity penalty coefficient is determined based on the logic break level. For subsequent stages where logical breaks occur consecutively, the value of the logical continuity penalty coefficient is progressively adjusted to decrease its value.

9. The state-driven interpretable evaluation system for 5E teaching materials as described in claim 1, characterized in that, Also includes: The evaluation status object module is used to construct a unified evaluation status object. The evaluation status object stores the evaluation input object, parsing results, stage target analysis results, logic break results, evidence pool, defect description, dimension scoring results, as well as preset parsing targets, stage targets, evaluation dimensions and scoring rules.

Images