Automatic review and personalized feedback system and method based on adaptive workflow

Abstract

The invention relates to the technical field of intelligent teaching, and discloses an automatic review and personalized feedback system and method based on adaptive workflow, and the system comprises a workflow engine, a metadata extraction module, a workflow engine operation module, a review module and a feedback module. The method corresponds to the system. According to the application, the workflow engine is combined with the metadata extraction module, so that the adaptability to multi-element scenes is improved; the review module integrates multiple types of review resources such as semantic comprehension and a knowledge graph, and realizes multi-dimensional accurate evaluation of subjective questions; the feedback module generates structured feedback, a personalized improvement path is provided in combination with knowledge point analysis, the learning efficiency and the depth of teaching feedback are remarkably improved, and the whole system breaks through the limitation of an existing review technology in the aspects of efficiency, fairness, accuracy and flexibility.

Description

Technical Field

[0001] The present application relates to the field of intelligent teaching technology, and specifically to an automatic review and personalized feedback system and method based on an adaptive workflow. Background Art

[0002] In existing teaching activities, there are significant technical bottlenecks in the homework review method of the Learning Management System (LMS).

[0003] Although manual grading can ensure a certain degree of accuracy and personalization, it is extremely inefficient. Teachers need to manually review subjective questions, which makes it difficult to cope with large-scale online education scenarios. In addition, the scoring criteria are greatly influenced by subjective factors, posing a risk to fairness. The feedback cycle is long, and it is impossible to stimulate students' enthusiasm for learning in a timely manner.

[0004] Simple automatic grading is only applicable to objective questions and relies on matching standard answers. For subjective questions, it can only use crude methods such as keyword matching. It cannot understand complex semantics, makes it difficult to evaluate the completeness, logic and innovation of the answers, and lacks in-depth analysis of the mastery of knowledge points.

[0005] Existing solutions, combined with the inefficiency of manual grading and the crudeness of automated grading in the context of subjective assessment, cannot meet the efficiency requirements of large-scale teaching and cannot provide accurate and personalized learning feedback. This results in an inability to effectively identify students' weaknesses and limits teaching effectiveness.

[0006] A Chinese invention patent application with publication number CN118505145A discloses an AI workflow design and execution device, method, and apparatus for teaching. However, the invention is poor in analyzing the workflow of teaching, especially homework review.

[0007] In summary, there is an urgent need for a new technical solution for automatic review and personalized feedback based on adaptive workflow. Summary of the Invention

[0008] The purpose of this application is to provide an automatic review and personalized feedback system and method based on adaptive workflow to solve the technical problems raised in the above background technology.

[0009] To achieve the above objectives, this application discloses the following technical solutions:

[0010] In a first aspect, the present application discloses an automatic review and personalized feedback system based on an adaptive workflow, the system comprising:

[0011] A workflow engine, configured to receive review materials, wherein the review materials are used to represent the content to be reviewed;

[0012] A metadata extraction module, configured to parse the review material and extract the material metadata;

[0013] A workflow engine running module, configured to utilize the workflow engine to analyze the material metadata and select or construct a review workflow;

[0014] A review module, configured to call a preset review resource based on the review workflow, review the review material using the review resource, and generate corresponding review results;

[0015] The feedback module is used to generate corresponding feedback materials based on the review results, wherein the feedback materials include the review results and content associated with the review results.

[0016] Preferably, the selection and review workflow in the selection or construction of the review workflow includes:

[0017] When there is a preset strategy ID in the review material, the corresponding review strategy is selected based on the strategy ID, the corresponding review resource is selected based on the review strategy, and the review workflow is selected based on the selected review resource; wherein, the strategy ID corresponds to the review strategy, the strategy ID is used to select the corresponding review strategy, and the review strategy is used to review the review material.

[0018] Preferably, the step of constructing a review workflow in the selection or construction of a review workflow includes:

[0019] When the strategy ID does not exist in the review material, the strategy ID is constructed using the material metadata, the corresponding review strategy is selected based on the constructed strategy ID, the corresponding review resource is selected, and the review workflow is constructed based on the selected review resource.

[0020] Preferably, the review resources include a semantic understanding unit, a knowledge graph association unit, a rule and keyword matching unit and a feature engineering unit.

[0021] Preferably, the workflow engine is further provided with an optimization module and a workflow configuration module;

[0022] The optimization module is used to generate an optimization strategy based on the review data corresponding to the historical review materials;

[0023] The workflow configuration module is used to optimize the review workflow based on the optimization strategy, build a strategy library based on the optimized review workflow, and store the strategy library in the workflow engine.

[0024] Preferably, the workflow configuration module is further configured to receive external review strategies.

[0025] Preferably, the content associated with the review results includes:

[0026] Based on the review result and the material metadata, a preset material library is called to match review materials associated with the material metadata, and the matched review materials are defined as content associated with the review result.

[0027] Preferably, a data preprocessing module is further provided for the metadata extraction module;

[0028] The data preprocessing module is used to perform data preprocessing on the review material.

[0029] Preferably, an output encapsulation module is further provided for the feedback module;

[0030] The output packaging module is used to package the feedback material and then output it.

[0031] In a second aspect, the present application discloses an automatic review and personalized feedback method based on an adaptive workflow, which is applicable to the automatic review and personalized feedback system based on an adaptive workflow as described above, and comprises:

[0032] S1: receiving review materials, wherein the review materials are used to represent the content to be reviewed;

[0033] S2: parsing the review material and extracting material metadata;

[0034] S3: Analyze the metadata of the material and select or build a review workflow;

[0035] S4: calling a preset review resource based on the review workflow, reviewing the review material using the review resource, and generating a corresponding review result;

[0036] S5: Generate corresponding feedback materials based on the review results, where the feedback materials include the review results and content associated with the review results.

[0037] Beneficial effects: The automatic review and personalized feedback system and method based on adaptive workflow of this application realizes the intelligence and precision of review process; the workflow engine is combined with metadata extraction module to dynamically select or construct review workflow according to review materials, flexibly adapt to differences in question types, subjects, etc., break through the rigidity of traditional fixed processes, and improve adaptability to multiple scenarios; the review module integrates multiple types of review resources such as semantic understanding and knowledge graphs, deeply analyzes the semantic logic and knowledge point coverage of students' answers, solves the extensive problem of traditional keyword matching, and realizes multi-dimensional and precise evaluation of subjective questions; the optimization module and workflow configuration module are combined to realize the intelligent and precise review ...s adaptability to multiple scenarios; the workflow engine is combined with metadata extraction module to dynamically select or construct review workflow according to review materials, flexibly adapt to differences in question types, subjects, etc., break through the rigidity of traditional fixed processes, and improves adaptability to multiple scenarios; the workflow engine is combined with metadata extraction module to dynamically select or construct review workflow according to review materials, The block supports the system's automatic evolution of review strategies based on historical data, and allows teachers to manually intervene when data is insufficient or strategy adjustments are needed, balancing machine intelligence and teaching experience; the data preprocessing module improves the quality of input data by cleaning answers and analyzing question features, laying the foundation for analysis; the output encapsulation module encapsulates review results in a standardized JSON format for easy integration and data traceability; the feedback module generates structured feedback and provides personalized improvement paths based on knowledge point analysis, significantly improving learning efficiency and the depth of teaching feedback. The overall system breaks through the limitations of existing review technologies in terms of efficiency, fairness, accuracy and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A structural diagram of the automatic review and personalized feedback system based on adaptive workflow provided in an embodiment of the present application;

[0040] Figure 2 This is a flowchart of the automatic review and personalized feedback method based on adaptive workflow provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following is a clear and complete description of the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] In this document, the term "comprising" is intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0043] In existing teaching activities, the existing learning management system (LMS) is usually used to review homework, which mainly includes the following methods:

[0044] Manual grading: Teachers manually review student assignments, especially subjective questions. This method ensures accurate and personalized grading, but it is time-consuming and inefficient, making it difficult to scale in large-scale online education scenarios. Teachers' grading standards may vary subjectively, leading to fairness issues. Furthermore, feedback is slow, which can dampen student motivation.

[0045] Simple automatic grading: This primarily targets objective questions (such as multiple-choice and fill-in-the-blank questions) by automatically matching answers to standard answers. For subjective questions, some systems may use simple methods such as keyword matching. However, this method struggles to accurately assess the completeness, logic, and innovation of answers, and its understanding of semantics is very limited.

[0046] In view of the above problems, the first aspect of this embodiment discloses Figure 1 An automatic review and personalized feedback system based on an adaptive workflow is shown, the system comprising:

[0047] A workflow engine is used to receive review materials, where the review materials are used to represent the content to be reviewed;

[0048] Metadata extraction module, used to parse review materials and extract metadata;

[0049] The workflow engine running module is used to analyze the metadata of the material using the workflow engine and select or build a review workflow;

[0050] The review module is used to call the preset review resources based on the review workflow, use the review resources to review the review materials, and generate corresponding review results;

[0051] The feedback module is used to generate corresponding feedback materials based on the review results, and the feedback materials include the review results and content associated with the review results.

[0052] It should be noted that the workflow engine in this embodiment communicates with the LMS, thereby receiving review materials and review requests initiated by the LMS. The LMS then provides feedback materials to students and teachers. In this embodiment, the workflow engine, metadata extraction module, workflow engine execution module, review module, and feedback module are sequentially connected in communication.

[0053] In this embodiment, the review materials may include:

[0054] Question information, including: question stem, question type, subject, difficulty, standard answer (if any), etc.

[0055] Student answers, including: text answers submitted by students.

[0056] Marking requirements include: for example, scoring criteria, key words, required arguments, answer structure requirements, etc.

[0057] Related knowledge points include: one or more knowledge point identifiers or descriptions that are strongly related to the topic.

[0058] The metadata extraction module is used in combination with existing data extraction technology to analyze the review materials and obtain the material metadata.

[0059] Through the above, compared with the existing LMS homework review method, the workflow engine drives automated review, which greatly reduces the time and labor-intensive problems of manual correction of subjective questions, and can cope with large-scale online education scenarios; standardized review resource calls avoid differences in teachers' subjective scoring and ensure fairness; the dynamic adaptation of the metadata extraction module and the review workflow enables the system to adaptively call review resources, break through the limitations of keyword matching, and achieve in-depth analysis of the semantics, logic and knowledge points of students' answers; the feedback module generates feedback materials based on the review results to provide students with targeted improvement suggestions and personalized learning paths, solving the problem of traditional feedback forms being rigid and lacking in depth; the workflow engine supports dynamic selection or construction of review processes based on review materials, and can flexibly adapt to the review needs of different subjects and question types, significantly improving the system's adaptability to diverse teaching scenarios.

[0060] With the development of AI technology, some LMS systems have begun to try to introduce AI technology for grading subjective questions, possibly utilizing basic natural language processing technology. However, these technologies often have the following shortcomings:

[0061] Limited semantic understanding capabilities: For complex sentence structures, synonym replacements, deep logical relationships, etc., AI may not be able to accurately grasp the true meaning of students' answers.

[0062] The grading dimension is single: most of the time, they only focus on the relevance of the content, and do not give enough consideration to the structure, logic, clarity of expression and other dimensions of the answer.

[0063] To address the above issues, this embodiment optimizes the workflow engine and workflow engine running module. Specifically, the selection or construction of the review workflow includes:

[0064] When there is a preset strategy ID in the review material, the corresponding review strategy is selected based on the strategy ID, the corresponding review resource is selected based on the review strategy, and the review workflow is selected based on the selected review resource; wherein, the strategy ID corresponds to the review strategy, the strategy ID is used to select the corresponding review strategy, and the review strategy is used to review the review material.

[0065] In the specific application of this embodiment, teachers or administrators can preset or customize review strategies and their corresponding strategy IDs based on subjects, question types, teaching objectives, etc. Each review strategy defines which review resources should be enabled under specific conditions, which model parameters to use, and the execution order or weight of each unit. For example:

[0066] Strategy ID "Strategy A" corresponds to the review strategy of "Chinese - Essay Questions - In-depth Analysis", specifically: enabling the Transformer semantic understanding unit and knowledge graph association unit, focusing on logical coherence and argument coverage.

[0067] Strategy ID "Strategy B" corresponds to the review strategy of "Mathematics - Short Answer Questions - Quick Verification", which specifically enables the rule and keyword matching unit and the traditional NLP feature engineering unit, focusing on the correctness of key steps and formulas.

[0068] The strategy ID "Strategy C" corresponds to the review strategy of "History - Multiple Choice Question Explanation - Knowledge Point Examination", specifically: enable the Transformer semantic understanding unit and focus on matching the association with core knowledge points.

[0069] The configured review strategy is stored in the strategy library for the workflow engine to call.

[0070] Through the above, the selection of review strategy and review resources is realized based on the preset strategy ID.

[0071] Specifically, the steps of constructing a review workflow in the selection or construction of a review workflow include:

[0072] When the strategy ID does not exist in the review material, the strategy ID is constructed using the material metadata, the corresponding review strategy is selected based on the constructed strategy ID, and the corresponding review resources are selected, and the review workflow is constructed based on the selected review resources.

[0073] In this embodiment, the workflow engine constructs a strategy ID based on the input source metadata and selects or dynamically constructs a suitable review workflow from the strategy library. This workflow defines the combination, sequence, and related parameters of the review resources to be executed in the subsequent review module. The strategy ID can be constructed based on preset rules or recommendations from the subsequent optimization module to select or dynamically construct a suitable review workflow.

[0074] In a specific application, the adaptive workflow of this embodiment specifically includes deterministic construction based on a rule engine, probabilistic recommendation based on a machine learning model, and real-time dynamic workflow combination.

[0075] The deterministic construction based on the rules engine is as follows: When review materials are received, the metadata extraction module analyzes their features. For example, if metadata.Subject = "Mathematics" AND metadata.Question Type = "Short Answer" AND metadata.Difficulty = "Medium", the rules engine will construct a specific strategy ID, such as "MATH_SA_MEDIUM". Based on this ID, a specific review workflow is then selected and orchestrated, which prioritizes the "Rule and Keyword Matching Unit" (for verifying formulas and key steps) and the "Feature Engineering Unit" (for assisting with scoring).

[0076] Probabilistic recommendations based on machine learning models utilize a pre-trained, offline multi-label classifier model (such as an existing decision tree or small neural network). This model takes the multi-dimensional metadata of the question (subject, question type, difficulty, list of related knowledge points, etc.) as an input vector and outputs one or a set of preset strategy IDs with the highest confidence. The highest confidence ID is selected to execute the corresponding workflow.

[0077] The specific real-time dynamic workflow combination is as follows: in the absence of an applicable policy ID, multiple review units (such as "semantic understanding unit" + "knowledge graph association unit") are dynamically selected and combined from the review resource pool based on metadata features to generate a review workflow with a temporary policy ID in real time.

[0078] Through the above, the adaptive construction of review strategies and review resources is achieved based on material metadata.

[0079] In this embodiment, the review module uses the review resource orchestrator to dynamically call one or more review resources to review the review materials according to the review workflow loaded by the workflow engine. The review resource orchestrator performs selective calling (for example, according to the workflow configuration, only the AI ​​units required for the current task are called, such as simple fill-in-the-blank questions may only require rule matching units, while complex essay questions may require a combination of Transformer and knowledge graph units), parameter passing (passing adaptive parameters to the called review resources, such as specific model versions, thresholds, etc.) and result integration (integrating analysis results from different review resources, such as semantic similarity scores, knowledge point coverage, rule matching flags, etc.) to form a comprehensive basis for evaluation.

[0080] In this embodiment, existing AI technology is used for review. Specifically, the review resources include a semantic understanding unit, a knowledge graph association unit, a rule and keyword matching unit, and a feature engineering unit.

[0081] It should be noted that, in this embodiment:

[0082] The semantic understanding unit can be a Transformer semantic understanding unit: using pre-trained Transformer models (such as BERT, RoBERTa, etc.) and their fine-tuned versions to understand the question intent and perform deep semantic analysis of student answers.

[0083] Knowledge graph association unit: Combined with the pre-built subject knowledge graph, it maps student answers and knowledge points, performs relationship reasoning, and coverage evaluation.

[0084] Rule and keyword matching unit: performs matching based on predefined rules, regular expressions or keyword lists, suitable for verifying specific patterns or factual content.

[0085] The feature engineering unit can be a traditional NLP feature engineering unit: extracting features such as the bag-of-words model, TF-IDF, and syntactic structure of the text, which can be used to assist in scoring or combined with other models.

[0086] The review resource of this embodiment may also be other dedicated AI units, for example, a unit for mathematical formula recognition, a static analysis unit for coding problems, etc.

[0087] In the application of this embodiment, for example, the selected review resources, such as the semantic understanding unit, deeply understand the test intent and core elements of the question and complete the question understanding; the selected review resources, such as the semantic understanding unit, perform in-depth semantic analysis on the pre-processed student answers, extract key arguments and logical structures, and complete the student answer parsing; the selected review resources or combination of review resources perform multi-dimensional matching and comparison of the parsed results of the student answers with the correction requirements to complete the correction requirement matching; further, if the workflow enables the knowledge graph unit or the rule and keyword matching unit, then the knowledge points associated with the question are combined to evaluate the degree of reflection and mastery of the student answers on these knowledge points, and complete the knowledge point association and evaluation;

[0088] In a simple example, dynamic semantic understanding enhancement based on the Transformer model:

[0089] In the review module, we dynamically select Transformer models of varying sizes or fine-tuned for specific domains based on the review workflow. For example, for quick preview scenarios, we can choose a lightweight Transformer model; for complex essay questions requiring high-precision analysis, we can use a larger, more sophisticated Transformer model.

[0090] The Transformer model is used to calculate text similarity or judge text implication. The strictness of the matching or the semantic dimension of attention can also be dynamically adjusted according to the workflow.

[0091] In another simple example, adaptive knowledge point evaluation and question recommendation based on knowledge graph:

[0092] The knowledge graph association unit and the rule and keyword matching unit dynamically load the corresponding subject knowledge graph according to the subject to which the question belongs and the workflow configuration.

[0093] The key concepts and arguments in students' answers are mapped to nodes in the knowledge graph. By analyzing these nodes and the relationships between them, the students' mastery of the knowledge points can be more accurately assessed.

[0094] The feedback module not only uses the knowledge graph to make recommendations based on students' weak knowledge points, but its recommendation strategy (such as whether to prioritize previous knowledge points or divergent related knowledge points) can also be dynamically configured by the workflow engine according to teaching objectives (such as consolidating the basics, expanding and advancing, and other different magic tables).

[0095] Through the above, review resources are used to provide the necessary technical means for automatic review and personalized feedback.

[0096] In the process of constructing the review workflow in this embodiment, a technical solution is designed to further optimize the construction process.

[0097] Specifically, the workflow engine is also provided with an optimization module and a workflow configuration module;

[0098] The optimization module is used to generate optimization strategies based on the review data corresponding to the historical review materials;

[0099] The workflow configuration module is used to optimize the review workflow based on the optimization strategy, build a strategy library based on the optimized review workflow, and store the strategy library in the workflow engine.

[0100] In this embodiment, the optimization module runs continuously in the background, collecting review data from historical review processes, which may include input question features, the review workflow used, review results, and the teacher's secondary proofreading / final score. Using existing machine learning algorithms (e.g., reinforcement learning, classification models, etc.), the optimization module analyzes the review effects of different workflow configurations in different scenarios (e.g., consistency with teacher scores, student subsequent learning performance, etc.). Based on the analysis results, optimization suggestions are made to the workflow configuration module, or the workflow engine's default strategy selection logic is automatically adjusted, thereby achieving continuous system adaptation and performance improvement.

[0101] In this embodiment, the specific process of optimization based on reinforcement learning is as follows:

[0102] Define the status as metadata corresponding to a review, such as [subject ID, question type ID, difficulty level, knowledge point A_ID, knowledge point B_ID];

[0103] Define actions to select each workflow that corresponds to a policy ID;

[0104] The reward is defined as a quantitative indicator used to measure the quality of the above action. The quantitative indicator is:

[0105]

[0106] When the system score is exactly the same as the teacher's final score, the reward is 1 (maximum), and the larger the gap, the lower the reward.

[0107] The learning process involves continuous review and reinforcement learning, which continuously explores the rewards obtained by performing different actions in different states. Leveraging existing Q-Learning algorithms or similar algorithms, the policy function is learned and optimized. The goal of this function is to automatically select the action stream with the highest expected reward for any given problem (i.e., state).

[0108] Based on the above reinforcement learning process, we have achieved continuous self-optimization of review and feedback based on dynamic adaptation according to the metadata of each review material, realizing adaptive review of different question types (subjective or objective questions), while ensuring the efficiency and consistency of large-scale applications.

[0109] Through the above, by adding optimization modules and workflow configuration modules, the system's dynamic evolution capability is built to achieve a closed loop of data, analysis and optimization, which not only improves the matching degree between the review workflow and complex scenarios, but also reduces the cost of manual intervention through machine learning, so that the system can continuously improve the review accuracy and teaching adaptability in the long-term operation, and enhance the level of intelligence.

[0110] It should be noted that when historical review materials are insufficient in the initial stage of system operation, or when there are major changes in the review strategy, teachers can directly configure the review strategy.

[0111] Specifically, the workflow configuration module is also used to receive external review strategies.

[0112] Through the above, based on the workflow configuration module, the limitations of the system in the initial data accumulation and the flexibility requirements for response strategy adjustments are compensated. Through manual intervention, the accuracy and adaptability of the review are guaranteed, and the organic combination of machine autonomous learning and teacher experience is realized.

[0113] The following problems were also found in the practice of using LMS for automatic review:

[0114] Lack of deep connection with knowledge points: It is difficult to accurately judge the degree of students' mastery of specific knowledge points.

[0115] Feedback is rigid and lacks personalization: it can usually only give scores and simple right or wrong judgments, and cannot provide targeted improvement suggestions. It is even more difficult to "draw inferences from one example" and help students consolidate their weak links.

[0116] The processing process is not transparent or rigid: There is a lack of unified and flexible workflow management for the inputs (such as questions, student answers, grading requirements, knowledge points) and outputs (such as grading results, recommended questions) in the grading process, making it difficult to adapt to the needs of different subjects and question types.

[0117] To address the above problem, this embodiment further generates content associated with the review results based on the review results.

[0118] Specific content related to the review results includes:

[0119] Based on the review results and material metadata, a preset material library is called to match review materials associated with the material metadata, and the matched review materials are defined as content associated with the review results.

[0120] In a simple example, the feedback module includes an answer correction and score generation unit and a "learn from one example and apply it to other situations" question generation / recommendation unit.

[0121] The answer correction and scoring unit generates an evaluation of the student's answer (e.g., comments, analysis of points scored, points lost) based on the evaluation criteria integrated into the review workflow, and assigns a final score or grade. Furthermore, the scoring itself can dynamically select or adjust weights based on the workflow configuration.

[0122] The "Draw Inferences from One Example" question generation / recommendation unit intelligently recommends relevant "Draw Inferences from One Example" exercises from a pre-set resource library (in this embodiment, the pre-set "Draw Inferences from One Example" question library) based on the student's weak knowledge points (identified by the review resources) or common error types revealed in the current test. The recommendation takes into account factors such as the difficulty of the question and the relevance to the weak knowledge points. Furthermore, the recommendation strategy (such as the number of recommendations and the difficulty gradient) is also configured by the workflow.

[0123] In this embodiment, the matching dimensions for the "learn from one example and apply it to other cases" question generation / recommendation unit to recommend questions include navigation based on the weak knowledge point map and attribution based on fine-grained error types.

[0124] Navigating weak knowledge points through the knowledge graph involves mapping the concepts identified in students' answers to knowledge graph nodes, pinpointing their weak points (e.g., a weak grasp of Newton's Second Law). Recommendations will then be limited to recommending questions from the resource library that specifically test that point and its predecessors or related points (e.g., force and acceleration).

[0125] Fine-grained error attribution involves categorizing errors based on the identification of correct and incorrect answers during review. For example, errors can be classified as "conceptual confusion," "calculation error," "logical gap," or "missing step." Based on the specific error type, questions are then matched to the most targeted training for that skill.

[0126] Through the above, by generating content related to the review results, we can achieve deep association of knowledge points and accurately locate students' weak links; the "learn from one example and apply it to other cases" question generation unit combines weak knowledge points and error types, and intelligently recommends matching questions from the question bank. When recommending, it takes into account factors such as difficulty and relevance. Moreover, the strategy can be dynamically configured through the workflow, breaking through the limitations of traditional feedback fixation, providing personalized improvement suggestions and targeted practice paths, helping students to find and fill in the gaps, and at the same time enhancing the flexibility and adaptability of the processing process through unified workflow management, thereby improving the accuracy and effectiveness of teaching reviews.

[0127] In practical applications, it is understandable that the data format of the review materials obtained using LMS may not necessarily meet the requirements of automatic review. Therefore, this embodiment uses existing data preprocessing technology to preprocess the data of the review materials.

[0128] Specifically, a data preprocessing module is also provided for the metadata extraction module;

[0129] The data preprocessing module is used to perform data preprocessing on the review materials.

[0130] In a simple example, student answers are cleaned, including removing irrelevant characters, formatting, sentence segmentation, stemming, and removing stop words. Question information and grading instructions are then structured and parsed to extract key features. Furthermore, data preprocessing itself can be adjusted based on the workflow engine's instructions. For example, different types of review resources may require different preprocessing methods.

[0131] Through the above, the data preprocessing module is used to provide targeted solutions to the problem of inconsistent review material data formats, thereby improving data availability, enhancing processing flexibility and optimizing review accuracy, laying the foundation for subsequent analysis, avoiding analytical deviations caused by rigid preprocessing and enhancing the overall review efficiency of the system.

[0132] In the specific application of this embodiment, after the feedback module generates feedback materials, it is necessary to feed back the feedback materials to students and teachers through the LMS. In response to this demand, this embodiment designs an output packaging module.

[0133] Specifically, an output encapsulation module is also provided for the feedback module;

[0134] The output packaging module is used to package the feedback material and output it.

[0135] In this embodiment, using existing encapsulation technology, the output encapsulation module encapsulates information such as corrected answers, scores, comments, weak knowledge point analysis, and content associated with the review results into a predefined, fixed-structure JSON object.

[0136] For example, the key fields of the JSON object include:

[0137] score: number, score

[0138] comment: string, overall comment

[0139] knowledge_point_assessment: object array (it should be noted that each object in this embodiment contains {kp_id, mastery_level, suggestions})

[0140] recommended_questions: object array (it should be noted that each object in this embodiment contains {question_id, reason_for_recommendation})

[0141] workflow_id_used: string, the workflow / strategy ID used for this review (it should be noted that the strategy ID is optional and its record in the JSON structure is used for tracing and analysis).

[0142] The generated JSON object is returned to the LMS, which is responsible for displaying it to students and teachers, thereby completing automatic review and personalized feedback based on adaptive workflow.

[0143] Through the above, feedback materials are encapsulated in standardized JSON format to achieve intuitive access by students and teachers, enhancing system compatibility and integration efficiency; traceable design provides data support for subsequent review effect analysis and strategy optimization, while reducing data processing complexity through unified output format, ensuring the accuracy and standardization of review results, significantly improving the system's data interaction and application efficiency, and improving the intelligence and efficiency of the teaching process.

[0144] The second aspect of this embodiment discloses Figure 2 The method for automatic review and personalized feedback based on an adaptive workflow is shown. The method is applicable to the automatic review and personalized feedback system based on an adaptive workflow as described above. The method includes:

[0145] S1: Receive review materials, which are used to represent the content to be reviewed;

[0146] S2: parse the review materials and extract the material metadata;

[0147] S3: Analyze material metadata and select or build review workflows;

[0148] S4: Based on the review workflow, the preset review resources are called, and the review materials are reviewed using the review resources to generate corresponding review results;

[0149] S5: Based on the review results, corresponding feedback materials are generated, and the feedback materials include the review results and content associated with the review results.

[0150] It should be noted that the adaptive workflow-based automatic review and personalized feedback method of this embodiment corresponds to the aforementioned adaptive workflow-based automatic review and personalized feedback system. Therefore, any content not specifically described in the adaptive workflow-based automatic review and personalized feedback method of this embodiment, including but not limited to functional definitions, operating principles, and technical effects, can be referred to in the aforementioned adaptive workflow-based automatic review and personalized feedback system and will not be elaborated upon in this document.

[0151] In summary, the automatic review and personalized feedback system and method based on adaptive workflow in this embodiment realizes the intelligence and precision of the review process; the workflow engine is combined with the metadata extraction module to dynamically select or construct the review workflow according to the review materials, flexibly adapt to the differences in question types, subjects, etc., break through the rigidity of traditional fixed processes, and improve adaptability to multiple scenarios; the review module integrates multiple types of review resources such as semantic understanding and knowledge graphs, deeply analyzes the semantic logic and knowledge point coverage of students' answers, solves the extensive problem of traditional keyword matching, and realizes multi-dimensional and precise evaluation of subjective questions; the optimization module and workflow configuration module It not only supports the system's automatic evolution of review strategies based on historical data, but also allows teachers to manually intervene when data is insufficient or strategy adjustments are needed, balancing machine intelligence and teaching experience; the data preprocessing module improves the quality of input data by cleaning answers and analyzing question features, laying the foundation for analysis; the output encapsulation module encapsulates review results in a standardized JSON format for easy integration and data traceability; the feedback module generates structured feedback and provides personalized improvement paths based on knowledge point analysis, significantly improving learning efficiency and the depth of teaching feedback. The overall system breaks through the limitations of existing review technologies in terms of efficiency, fairness, accuracy and flexibility.

[0152] In the embodiments provided herein, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code, or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a controller, a microcontroller, a microprocessor, other electronic units designed to implement the functions described herein, or a combination thereof. For software implementation, part or all of the processes of the embodiments can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein the communication media include any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a computer can access. The computer-readable storage medium can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0153] Finally, it should be noted that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent replacements for some of the technical features therein. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. An automatic review and personalized feedback system based on adaptive workflow, characterized by: The system includes: A workflow engine, configured to receive review materials, wherein the review materials are used to represent the content to be reviewed; A metadata extraction module, configured to parse the review material and extract the material metadata; A workflow engine running module, configured to utilize the workflow engine to analyze the material metadata and select or construct a review workflow; A review module, configured to call a preset review resource based on the review workflow, review the review material using the review resource, and generate corresponding review results; The feedback module is used to generate corresponding feedback materials based on the review results, wherein the feedback materials include the review results and content associated with the review results.

2. The automatic review and personalized feedback system based on adaptive workflow according to claim 1 is characterized in that: The selection or construction review workflow includes: When there is a preset strategy ID in the review material, the corresponding review strategy is selected based on the strategy ID, the corresponding review resource is selected based on the review strategy, and the review workflow is selected based on the selected review resource; wherein, the strategy ID corresponds to the review strategy, the strategy ID is used to select the corresponding review strategy, and the review strategy is used to review the review material.

3. The automatic review and personalized feedback system based on adaptive workflow according to claim 2, characterized in that: The construction of the review workflow in the selection or construction of the review workflow includes: When the strategy ID does not exist in the review material, the strategy ID is constructed using the material metadata, the corresponding review strategy is selected based on the constructed strategy ID, the corresponding review resource is selected, and the review workflow is constructed based on the selected review resource.

4. The automatic review and personalized feedback system based on adaptive workflow according to claim 1, characterized in that: The review resources include a semantic understanding unit, a knowledge graph association unit, a rule and keyword matching unit, and a feature engineering unit.

5. The automatic review and personalized feedback system based on adaptive workflow according to claim 1 is characterized in that: The workflow engine is also provided with an optimization module and a workflow configuration module; The optimization module is used to generate an optimization strategy based on the review data corresponding to the historical review materials; The workflow configuration module is used to optimize the review workflow based on the optimization strategy, build a strategy library based on the optimized review workflow, and store the strategy library in the workflow engine.

6. The automatic review and personalized feedback system based on adaptive workflow according to claim 5, characterized in that: The workflow configuration module is further configured to receive external review strategies.

7. The automatic review and personalized feedback system based on adaptive workflow according to claim 1 is characterized in that: The content associated with the review results includes: Based on the review result and the material metadata, a preset material library is called to match review materials associated with the material metadata, and the matched review materials are defined as content associated with the review result.

8. The automatic review and personalized feedback system based on adaptive workflow according to claim 1, characterized in that: A data pre-processing module is also provided for the metadata extraction module; The data preprocessing module is used to perform data preprocessing on the review material.

9. The automatic review and personalized feedback system based on adaptive workflow according to claim 1, characterized in that: An output packaging module is also provided for the feedback module; The output packaging module is used to package the feedback material and then output it.

10. An automatic review and personalized feedback method based on an adaptive workflow, the method being applicable to the automatic review and personalized feedback system based on an adaptive workflow according to any one of claims 1 to 9, characterized in that: The method includes: S1: receiving review materials, wherein the review materials are used to represent the content to be reviewed; S2: parsing the review material and extracting material metadata; S3: Analyze the metadata of the material and select or build a review workflow; S4: calling a preset review resource based on the review workflow, reviewing the review material using the review resource, and generating a corresponding review result; S5: Generate corresponding feedback materials based on the review results, where the feedback materials include the review results and content associated with the review results.

Images