Student actual combat homework AI intelligent evaluation method, system and device based on hybrid expert model and storage medium
By using an AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model, the problem of inconsistent scoring standards in evaluations of multiple question types and courses is solved. This method achieves the verifiability and interpretability of evaluation results, ensuring the comparability between different assignments and the stability of the total score.
Patent Information
- Application Number
- CN202512013663.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-05-01
AI Technical Summary
Existing technologies make it difficult to achieve unified, consistent, verifiable, and interpretable scoring standards for multiple question types and courses in the evaluation of practical assignments for college and vocational education students. In particular, when scoring directly in large models and outputting results from static analysis tools, there is a lack of stable integration and boundary handling mechanisms, which makes it difficult to structure the evaluation results and makes them less comparable.
An AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model is adopted. This method obtains the content of student assignments and their teaching task identifiers, reads pre-constructed structured expected answers, determines the subset of experts for the dimensions to be activated, calls the artificial intelligence model to generate structured evaluation outputs, and obtains the final score results through predefined calculation rules and weight normalization fusion. Combined with the lowest score protection and threshold penalty mechanism, the consistency and stability of the evaluation are ensured.
It achieves consistency and comparability of evaluation results across multiple question types and course scenarios, ensures the verifiability and interpretability of evaluation results, avoids drift in the total score scale, and provides stable structured scoring results.
Smart Images
Figure CN121961790A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of educational informatization and artificial intelligence technology, and more specifically, to an AI-based intelligent evaluation method, system, device, and storage medium for student practical assignments based on a Mixture of Experts (MoE) model. Background Technology
[0002] In the teaching scenario of "student practical assignments" in higher education and vocational education, the evaluation objects not only include traditional subjective questions, but also often include multi-form results such as code, project descriptions, experimental processes and results, which places higher demands on the consistency, verifiability and scalability of the evaluation. Existing assignment evaluation systems usually adopt methods such as manual grading, rule-based scoring or direct scoring by large models. However, in actual operation with multiple question types and multiple courses, there are common problems such as difficulty in unifying scoring standards, difficulty in structuring and verifying evaluation results, and difficulty in stabilizing the total score scale when the applicability of different evaluation dimensions changes. Therefore, there is an urgent need for an intelligent evaluation engine that can break down the evaluation task into multi-dimensional expert evaluations and achieve stable and interpretable total score output through gating activation and fusion computing.
[0003] One of the mainstream technical approaches for "intelligent evaluation of student practical assignments" is to run student code through online evaluation / sandbox execution (such as OJ / evaluation server), and use test cases and performance metrics to form evaluation results. Evaluation results heavily rely on the consistency of the executable environment, requiring the maintenance of multi-language runtimes, dependencies, and isolation strategies. Simultaneously, performance metrics are affected by environment and resource scheduling, leading to insufficient comparability in batch evaluations or multi-node deployments, and making it difficult to output structured explanatory information oriented towards teaching objectives (such as deduction points, missing points, and improvement suggestions).
[0004] Another common approach is to use static analysis tools to check code for rule compliance (standards / security / style, etc.). Static analysis output is usually a list of alerts, which is difficult to map naturally to teaching scoring details such as "dimensional scoring - deduction level - weight fusion - minimum score protection"; at the same time, static analysis is difficult to provide "efficiency scores and explanations" that are directly related to teaching objectives, thus making it difficult to form a unified structured scoring result that can be used for teaching diagnosis.
[0005] In terms of multi-dimensional evaluation, some systems use fixed dimensions / fixed weights or simple normalization to summarize the total score. Different question types have different applicability to dimensions, and fixed dimensions will introduce noise from inapplicable dimensions. When the system can only execute or select to execute some dimensions, traditional integration methods often lack targeted weight normalization and comparability constraints, which can easily cause the total score scale to drift with changes in the activation set, making it difficult to support stable comparison and tracking across question types and courses.
[0006] Furthermore, while the "direct scoring based on prompts / free text comments" scheme based on large models can cover subjective questions, its technical shortcomings lie in the fact that the output often lacks parsable and database-compatible structured evidence fields (such as error lists, missing item lists, sets of hit knowledge points, etc.), and it lacks stable fusion and boundary handling mechanisms (threshold penalties, minimum score protection, etc.), making it difficult to guarantee scoring consistency and verifiability in engineering. Therefore, existing technologies cannot simultaneously satisfy the following evaluation engine closed loops: dynamically activating applicable dimensions in multi-question scenarios, ensuring consistency and comparability of total score scales, and outputting structured and interpretable results. Summary of the Invention
[0007] The technical problem this invention aims to solve is that the industry commonly uses "LLM + prompt word scoring" or "Rubric scoring table + LLM scoring" methods, as described in the background art, with some products introducing multi-dimensional Rubric and result interpretation. However, in terms of the complete engineering closed loop of "dynamic expert activation (gating) + structured scoring output + weight normalization fusion + penalty / lower limit protection + conflict calibration," there are still problems such as complex implementation, insufficient stability, or weak configurability. To solve the above problems, this invention provides an AI intelligent evaluation method, system, device, and storage medium for student practical assignments based on a hybrid expert model.
[0008] The object of this invention is achieved in the following manner: An AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model, the method comprising: Obtain the content of student-submitted assignments and the task identifiers of the teaching tasks they belong to; Based on the task identifier, read the pre-constructed structured expected answer corresponding to the teaching task, wherein the structured expected answer contains at least dimensional activation information; Based on the aforementioned dimension activation information, determine the subset of dimension experts that need to be activated in this evaluation; For each dimension expert in the aforementioned subset of dimension experts, an artificial intelligence model is invoked to generate a structured evaluation output for that dimension; Based on predefined calculation rules, the structured evaluation output for each dimension is parsed and calculated to obtain the dimension score for each dimension. The original weights corresponding to each dimension in the dimensional expert subset are normalized to obtain normalized weights; The dimensional scores are weighted and fused according to the normalized weights to obtain the final score of the task.
[0009] The structured expected answer also includes at least one of the following: a list of required items and a list of optional items for the completeness dimension; an error level deduction mapping rule for the correctness dimension; and a set of expected knowledge points for the knowledge point matching dimension.
[0010] The step of determining the subset of dimension experts to be activated based on the dimension activation information specifically involves: Parse the expert activation field in the structured expected answer, where the expert activation field is a boolean dictionary or an array containing activation states; Add the dimension experts whose activation status is marked in the expert activation field to the dimension expert subset.
[0011] The original weights corresponding to each dimension in the dimensional expert subset are normalized using the following formula: in, E active This represents the subset of experts in the stated dimension. w e Representation Dimension Expert e The original weights, w′ e This indicates its normalized weight.
[0012] The dimensional score is obtained based on predefined calculation rules, including at least one of the following methods: For the completeness dimension, a base score is calculated based on the matching rate between the number of required and optional items completed by students and the expectation list, and a penalty coefficient is applied to the matching rate that does not reach the threshold. For the correctness dimension, points are deducted based on the number of errors of different levels in the structured evaluation output and predefined deduction values, and the lowest score protection is applied. For the knowledge point matching dimension, the Jakarta similarity coefficient between the set of knowledge points matched by the student's assignment and the expected set of knowledge points is calculated, and this similarity is mapped to the dimension score.
[0013] After obtaining the homework submitted by students, the method further includes: Verify the construction status of the structured expected answer corresponding to the teaching task; If the construction status is incomplete, the evaluation process will be terminated and a status message will be returned.
[0014] The process of calling the artificial intelligence model to generate a structured evaluation output for this dimension specifically includes: Based on the evaluation objectives of the aforementioned dimensions, construct prompt information that includes the task content and relevant content from the structured expected answer; The prompt information is input into the artificial intelligence model, and the artificial intelligence model is constrained to output structured results in JSON format.
[0015] An AI-powered intelligent evaluation system for student practical assignments based on a hybrid expert model includes: Input processing module: used to obtain the content of the assignments submitted by students and the task identifier of the teaching task to which they belong; Expected Answer Management Module: Used to store and manage pre-built structured expected answers corresponding to each teaching task, wherein the structured expected answers contain at least dimensional activation information; Gating and expert activation module: used to read the corresponding structured expected answer from the expected answer management module according to the task identifier, and determine the subset of dimension experts to be activated for this evaluation based on the dimension activation information therein; Multi-dimensional expert evaluation module: includes multiple dimension expert units, used to call the corresponding artificial intelligence model according to the instructions of the gating and expert activation module, and generate structured evaluation output for each dimension of the dimension expert subset; The fusion scoring module is used to convert the structured evaluation output of each dimension into dimension scores based on predefined calculation rules, and then perform weighted fusion after normalizing the weights of the expert subsets of the dimensions to obtain the final scoring result. Results output module: Used to output and store structured evaluation results containing the final score and details of each dimension.
[0016] An electronic device, comprising: One or more processors; Memory; One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, the one or more computer programs including instructions for performing the method.
[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method.
[0018] The beneficial effects of this invention are as follows: By pre-constructing and solidifying the "expected answer" for each task, this invention places the scoring criteria and dimensional strategies in advance, ensuring that all student assignments are measured under the same "ruler," and that the evaluation results are verifiable and traceable. The introduction of a fusion strategy that normalizes the weights of activated dimensions mathematically guarantees that the theoretical range of the final total score remains stable regardless of which dimensions are activated, making the scores of different assignments and question types comparable. Simultaneously, combined with mechanisms such as minimum score protection and threshold penalties, it effectively smooths out random fluctuations in the AI model output. Attached Figure Description
[0019] Figure 1 This is a flowchart of an AI-based intelligent evaluation method for student practical assignments based on a hybrid expert model (MoE) provided in an embodiment of the present invention.
[0020] Figure 2 This is a block diagram of an AI-powered intelligent evaluation system for student practical assignments based on a hybrid expert model (MoE) according to an embodiment of the present invention.
[0021] Figure 3 This is a schematic diagram illustrating the principle of multi-dimensional expert evaluation and weight normalization fusion based on gating activation in this invention. Detailed Implementation
[0022] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same technical meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0024] This invention discloses an AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model, the method comprising: Obtain the assignment content submitted by students and the task identifier of the teaching task to which it belongs; the task identifier is a unique task number (ID) or a composite identifier composed of course identifier and assignment sequence number.
[0025] Based on the task identifier, read the pre-constructed structured expected answer corresponding to the teaching task, wherein the structured expected answer contains at least dimensional activation information; Based on the aforementioned dimension activation information, determine the subset of dimension experts that need to be activated in this evaluation; For each dimension expert in the aforementioned subset of dimension experts, an artificial intelligence model is invoked to generate a structured evaluation output for that dimension; Based on predefined calculation rules, the structured evaluation output for each dimension is parsed and calculated to obtain the dimension score for each dimension. The original weights corresponding to each dimension in the dimensional expert subset are normalized to obtain normalized weights; The dimensional scores are weighted and fused according to the normalized weights to obtain the final score of the task.
[0026] The structured expected answer also includes at least one of the following: a list of required items and a list of optional items for the completeness dimension; an error level deduction mapping rule for the correctness dimension; and a set of expected knowledge points for the knowledge point matching dimension.
[0027] The step of determining the subset of dimension experts to be activated based on the dimension activation information specifically involves: Parse the expert activation field in the structured expected answer, where the expert activation field is a boolean dictionary or an array containing activation states; Add the dimension experts whose activation status is marked in the expert activation field to the dimension expert subset.
[0028] The original weights corresponding to each dimension in the dimensional expert subset are normalized using the following formula: Among them, E active Denotes the dimensional expert subset, w e w represents the original weight of dimension expert e, and w′e represents its normalized weight.
[0029] The dimensional score is obtained based on predefined calculation rules, including at least one of the following methods: For the completeness dimension, a base score is calculated based on the matching rate between the number of required and optional items completed by students and the expectation list, and a penalty coefficient is applied to the matching rate that does not reach the threshold. For the correctness dimension, points are deducted based on the number of errors of different levels in the structured evaluation output and predefined deduction values, and the lowest score protection is applied. For the knowledge point matching dimension, the Jakarta similarity coefficient between the set of knowledge points matched by the student's assignment and the expected set of knowledge points is calculated, and this similarity is mapped to the dimension score.
[0030] After obtaining the homework submitted by students, the method further includes: Verify the construction status of the structured expected answer corresponding to the teaching task; If the construction status is incomplete, the evaluation process will be terminated and a status message will be returned.
[0031] The process of calling the artificial intelligence model to generate a structured evaluation output for this dimension specifically includes: Based on the evaluation objectives of the aforementioned dimensions, construct prompt information that includes the task content and relevant content from the structured expected answer; The prompt information is input into the artificial intelligence model, and the artificial intelligence model is constrained to output structured results in JSON format.
[0032] According to another aspect of the present invention, an AI-powered intelligent evaluation system for student practical assignments based on a hybrid expert model is provided, comprising: Input processing module: used to obtain the content of the assignments submitted by students and the task identifier of the teaching task to which they belong; Expected Answer Management Module: Used to store and manage pre-built structured expected answers corresponding to each teaching task, wherein the structured expected answers contain at least dimensional activation information; Gating and expert activation module: used to read the corresponding structured expected answer from the expected answer management module according to the task identifier, and determine the subset of dimension experts to be activated for this evaluation based on the dimension activation information therein; Multi-dimensional expert evaluation module: includes multiple dimension expert units, used to call the corresponding artificial intelligence model according to the instructions of the gating and expert activation module, and generate structured evaluation output for each dimension of the dimension expert subset; The fusion scoring module is used to convert the structured evaluation output of each dimension into dimension scores based on predefined calculation rules, and then perform weighted fusion after normalizing the weights of the expert subsets of the dimensions to obtain the final scoring result. Results output module: Used to output and store structured evaluation results containing the final score and details of each dimension.
[0033] An electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs include instructions for performing the method.
[0034] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method.
[0035] Example: As attached Figure 3 As shown, this invention determines the set of experts for the activated dimensions through a gating mechanism, and only normalizes and fuses the weights of the activated dimensions, thereby maintaining the stability and comparability of the total score scale even when the activated dimensions of different tasks are different.
[0036] 1) Construct a "task-level evaluation benchmark (ExpectedAnswer)" and solidify the gating results. For each teaching task / assignment question, the system first generates and saves an "Expected Answer / Evaluation Criteria," which includes at least: (1) The “expectation list / standards” required for scoring each dimension (e.g., required / optional items, expected knowledge points, and description of scoring rules); (2) Expert activation set experts_activation: refers to which dimension experts need to be activated when evaluating this task (e.g., completeness / correctness / standardization / knowledge point matching / efficiency).
[0037] By embedding experts_activation into the task-level ExpectedAnswer, "gated activation" becomes traceable and consistent, avoiding the need to re-determine "whether a certain dimension is activated" for each evaluation.
[0038] 2) Pre-evaluation condition check: Ensure evaluation criteria are in place. For any student assignment record Job, before evaluation, the system checks whether the expected answer status of the task has been completed. If not, the evaluation process is not initiated, and a clear status of "expected answer not ready / cannot be evaluated" is returned to ensure consistency of evaluation input conditions.
[0039] 3) Activate the dimensional experts by pressing experts_activation and generate structured output. During the evaluation phase, the `experts_activation` function in `ExpectedAnswer` is read to obtain the set of activation dimensions `E`. active .
[0040] right For each dimension, experts use the provided template to generate structured output (JSON / parsable text) from the model, instead of free text scores, to ensure that the output is computationally calculable, can be stored in a database, and can be verified.
[0041] 4) Map the structured output to dimensional scores (including stability processing) The system parses and performs rule-based calculations on the structured output for each dimension to obtain the dimension score. With explanatory information (deduction points / missing points / hit sets / comments).
[0042] Stability mechanisms such as "threshold penalty, minimum score protection, and boundary truncation" are introduced to prevent extreme outputs and scale drift.
[0043] 5) Only the activation dimensions are weighted and normalized to obtain the total score. To address the issue of incomparable total scores when only some experts are activated, the system only sums and normalizes the weights of the activated dimensions: Calculate the total score: Output structured overall evaluation results It includes: total score, scores for each dimension, explanatory information and (optional) report links, and is written into the database for the platform to display and trace.
[0044] (a) System Inputs and Key Data Objects Assignment Input A: Student-submitted assignment text / answer content / attachment reference information. Attachments can include images, documents, PDFs, etc.; OCR for images / documents is only used to support input standardization.
[0045] Evaluation Configuration C: Weights for each dimension, threshold parameters, deduction mapping, minimum score protection, etc. (configurable).
[0046] ExpectedAnswer: Task-level evaluation benchmark, including experts_activation, a list of required / optional tasks, a set of expected knowledge points, and a text or structured fragment of scoring rules.
[0047] (II) Methods, Steps and Sequence of Actions As attached Figure 1 As shown, the AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model of the present invention includes the following steps S1 to S7: S1: Get job and task configuration: Read the job record, task record and evaluation configuration corresponding to jobId.
[0048] S2: Precondition checks: Verify whether the job has been submitted; verify whether expected_answer_status is "Build completed". If not met, terminate and return a specific error code / status.
[0049] S3: Read ExpectedAnswer and extract the gating result: Parse ExpectedAnswer to obtain: experts_activation (structured field for the activation dimension, which can be a boolean dictionary or an array of key / label / active); Expected data required for each dimension of scoring (such as expected_required_items, expected_optional_items, expected_knowledge_points, etc.).
[0050] S4: Perform expert evaluation by activation set: for each Construct the system prompt and user prompt for this dimension (injecting the job content and corresponding expectation rules), and call the model to obtain the raw output raw_e.
[0051] S5: Parse the structured output and calculate the dimensional scores: Use different parsing and calculation methods for different dimensions (see below).
[0052] S6: Fusion Score: The total score is obtained by weighted summation after normalizing the weights of the activation dimensions, and necessary boundary processing is performed.
[0053] S7: Output and Archiving: Write to the database (dimension details, total score, comments, optional report address) and return to the front end for display; report generation can be used as post-processing, or it can be written to the database in the same process as the scoring results. If it fails, roll back or retry.
[0054] (III) Structured Output and Calculation Rules for Scores in Each Dimension 1) Completeness Expert Structured output requirements: Output required_items (list of completed required items), optional_items (list of completed optional items), and comment.
[0055] Calculation process: The system matches the output list with the expected list and calculates the completion rate. required_rate = (Number of required items completed / Total number of expected required items) × 100 optional_rate = (Number of completed options / Total number of expected options) × 100 Scoring formula: When required_rate < 60%, Base = Base × 0.7 (threshold penalty); the final score is rounded to the nearest integer.
[0056] Effect description: The evaluation criteria are verifiable through a structured list of "required / optional items"; The threshold penalty mechanism ensures that "missing key points" will consistently lower the score, preventing the model from only giving positive reviews without deducting points.
[0057] 2) Correctness Expert Structured output requirements: Output critical_errors, major_errors, normal_errors, trivial_errors (four-level error list) and comments.
[0058] Deduction calculation: Let the number of Level 4 errors be... If the deduction mapping is 8 / 5 / 2 / 1, then: Effect description: Verifiable evidence of point deductions is generated through a combination of "error level + error list"; Minimum score protection (20 points) avoids extreme output and improves stability.
[0059] 3) Knowledge Matching Expert Structured output requirements: Output used_knowledge_points (the set of knowledge points hit by the assignment) and comments.
[0060] Calculation rule: Expected knowledge point set Hit set The Jaccard similarity (Jaccardindex: intersection-union ratio of sets) is used: Dimensional Score: Effect description: Stable and comparable quantitative indicators are obtained through set operations; Develop a structured evaluation basis oriented towards "teaching objectives / knowledge coverage".
[0061] 4) Normative Expert and Efficiency Expert Structured output requirements: Output score (0-100 recommended) and comment.
[0062] Processing rules: scores can be truncated from 0 to 100 or constrained by prompt words; minimum score protection can be configured by dimension.
[0063] Effect description: A unified dimension output interface facilitates the addition of new dimensions and the replacement of models; Dimensions can be enabled / disabled according to task configuration to avoid interference from unsuitable dimensions.
[0064] 5) Integration score Only for After normalizing the dimensional weights, they are merged to avoid "the total score scale drift caused by the activation of dimensional subsets".
[0065] Effect description: Addressing the issue of varying applicability of dimensions in scenarios with multiple question types; This ensures that the total scores remain comparable across different assignments / tasks, even if the sets of activated dimensions are different.
[0066] (iv) Abnormal branches and fault tolerance mechanisms If the model output cannot be parsed as JSON or is missing a key field: record an alert and retry according to the configuration; if it still fails, the dimension will be handled according to the downgrade strategy (such as conservative score / default score / 0 score, the strategy is configurable), but it will not affect the calculation of other dimensions and the total score.
[0067] Some dimensions are unavailable / disabled: Fusion only uses successfully activated dimensions and normalizes their weights to avoid "one-vote veto".
[0068] If the expected answer is missing or incomplete: Return directly to the "Expected Answer Not Ready" status, prompting you to build ExpectedAnswer first.
[0069] like Figure 2 As shown, this invention provides an AI-powered intelligent evaluation system for student practical assignments based on a hybrid expert model (MoE), including but not limited to an input processing module, an expected answer construction module, a gating and expert activation module, a multi-dimensional expert evaluation module, a fusion scoring module, and a result output and archiving module. The input processing module is used to acquire assignment content, task configuration, and submitter information and generate standardized input. Expected Answer Construction Module: Used to generate / maintain expected answers and scoring criteria, key point list, knowledge point set, and generate expert activation information; Gating and Expert Activation Module: Used to determine the set of experts for the dimensions to be activated in this evaluation, and to normalize the weights of the activated dimensions. Multi-dimensional expert evaluation module (sub-domain expert module): used to output structured evaluation results for dimensions such as completeness, correctness, standardization, knowledge point matching, and (optional) efficiency; The fusion scoring module is used to normalize the scores of each dimension and calculate the total score by weighting and fusion, and to perform threshold penalty, minimum score protection and anomaly fault tolerance. Results Output and Archiving Module: Used to output structured overall evaluation results, write them to storage, and display them on the front end; optional, it can generate evaluation report files; Batch processing / scheduling module (optional): Used for batch triggering, scheduled execution, and task management of expected answer construction and job evaluation.
[0070] To enhance the scope of inventiveness and prevent circumvention, the present invention may further include the following improvements based on the above-described basic concept: 1) Improved gating strategies (from fixed activation to learnable activation) In the current embodiment, the gating result comes from the experts_activation during the expected answer construction phase.
[0071] Alternatively, the activation set can be dynamically adjusted based on experts_activation, taking into account historical consistency indicators, teacher review results, course difficulty, and other information; or a two-level gating system can be introduced (first coarsely select dimensions, then finely select expert strategies).
[0072] Beneficial effects: Reduces false activations / missed activations, further improving stability and efficiency.
[0073] 2) Improved fusion strategies (robust fusion / confidence-weighted fusion) Based on the weighted average, the following can be added: extreme value removal fusion, quantile fusion, confidence weighting (if a quality hint field is implemented later), or an "arbitration expert" can be introduced to re-evaluate the conflict dimensions.
[0074] Beneficial effects: Suppresses the impact of abnormal dimension outputs on the total score, improving robustness.
[0075] 3) The dimensional system is pluggable and expandable (subject-based / task-based experts). In addition to the general dimensions (completeness / correctness / standardization / knowledge point matching), new expert dimensions can be added according to discipline or task, such as: argumentation quality, procedural rigor, and logical expression; simply add: expert prompt template + structured parsing rules + weight configuration.
[0076] Beneficial effects: Adapts to more professional and course objectives, enhancing scalability.
[0077] 4) Enhancement of evaluation evidence (evidence indexing, visual annotation) The structured output can further include evidence fragment location indexes (character ranges / paragraph numbers in the original work text) for front-end highlighting and quick review.
[0078] Beneficial effects: Improves interpretability and verification efficiency, and reduces the cost of secondary confirmation for teachers.
[0079] 5) Optional enhancements to input standardization (not considered as the main inventive point) Text information can be extracted from images / PDFs / documents using OCR or a multimodal model as supplementary input for A; OCR is an optional preprocessing capability and does not affect the core of the evaluation engine.
[0080] Beneficial effects: Improves the evaluability of complex submissions, but does not change the main theme of "multi-expert evaluation + integration".
[0081] 6) Batch processing / scheduled execution (project implementation and expansion) A timed / batch processing module can be added to batch trigger and task orchestration for ExpectedAnswer construction and job evaluation.
[0082] Beneficial effects: It meets the needs of large-scale evaluation in scenarios involving large classes and multiple tasks in parallel.
[0083] This invention features consistent and traceable evaluation: by pre-constructing and solidifying the "expected answer" for each task, the scoring criteria and dimensional strategies are brought forward, ensuring that all student assignments are measured under the same "ruler," and the evaluation results are verifiable and traceable.
[0084] This invention offers scoring stability and comparability: it introduces a fusion strategy that normalizes the weights of activated dimensions only, mathematically guaranteeing that the theoretical range of the final total score remains stable regardless of which dimensions are activated, thus making scores from different assignments and question types comparable. Simultaneously, combined with mechanisms such as minimum score protection and threshold penalties, it effectively mitigates random fluctuations in the AI model's output.
[0085] This invention features an interpretable evaluation process: it employs a "structured evidence output + deterministic rule calculation" model, limiting the role of the AI model to "evidence extraction," while the score is calculated using transparent and deterministic rules. This ensures that every deduction and every score is verifiable, forming a complete chain of scoring evidence.
[0086] This invention features a flexible and scalable system: the MoE-based gating architecture and pluggable dimensional expert design enable the system to easily adapt to new courses and new evaluation dimensions. Evaluation strategies can be quickly switched by configuring different experts_activation settings.
[0087] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several changes and improvements without departing from the overall concept of the present invention, and these should also be considered within the scope of protection of the present invention.
Claims
1. An AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model, characterized in that: The method includes: Obtain the content of student-submitted assignments and the task identifiers of the teaching tasks they belong to; Based on the task identifier, read the pre-constructed structured expected answer corresponding to the teaching task, wherein the structured expected answer contains at least dimensional activation information; Based on the aforementioned dimension activation information, determine the subset of dimension experts that need to be activated in this evaluation; For each dimension expert in the aforementioned subset of dimension experts, an artificial intelligence model is invoked to generate a structured evaluation output for that dimension; Based on predefined calculation rules, the structured evaluation output for each dimension is parsed and calculated to obtain the dimension score for each dimension. The original weights corresponding to each dimension in the dimensional expert subset are normalized to obtain normalized weights; The dimensional scores are weighted and fused according to the normalized weights to obtain the final score of the task.
2. The AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model as described in claim 1, characterized in that: The structured expected answer also includes at least one of the following: a list of required items and a list of optional items for the completeness dimension; an error level deduction mapping rule for the correctness dimension; and a set of expected knowledge points for the knowledge point matching dimension.
3. The AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model as described in claim 1, characterized in that: The step of determining the subset of dimension experts to be activated based on the dimension activation information specifically involves: Parse the expert activation field in the structured expected answer, where the expert activation field is a boolean dictionary or an array containing activation states; Add the dimension experts whose activation status is marked in the expert activation field to the dimension expert subset.
4. The AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model as described in claim 1, characterized in that: The original weights corresponding to each dimension in the dimensional expert subset are normalized using the following formula: in, E active This represents the subset of experts in the stated dimension. w e Representation Dimension Expert e The original weights, w′ e This indicates its normalized weight.
5. The AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model as described in claim 1, characterized in that: The dimensional score is obtained based on predefined calculation rules, including at least one of the following methods: For the completeness dimension, a base score is calculated based on the matching rate between the number of required and optional items completed by students and the expectation list, and a penalty coefficient is applied to the matching rate that does not reach the threshold. For the correctness dimension, points are deducted based on the number of errors of different levels in the structured evaluation output and predefined deduction values, and the lowest score protection is applied. For the knowledge point matching dimension, the Jakarta similarity coefficient between the set of knowledge points matched by the student's assignment and the expected set of knowledge points is calculated, and this similarity is mapped to the dimension score.
6. The AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model as described in claim 1, characterized in that: After obtaining the homework submitted by students, the method further includes: Verify the construction status of the structured expected answer corresponding to the teaching task; If the construction status is incomplete, the evaluation process will be terminated and a status message will be returned.
7. The AI-powered intelligent evaluation method for student practical assignments based on a hybrid expert model as described in claim 1, characterized in that: The process of calling the artificial intelligence model to generate a structured evaluation output for this dimension specifically includes: Based on the evaluation objectives of the aforementioned dimensions, construct prompt information that includes the task content and relevant content from the structured expected answer; The prompt information is input into the artificial intelligence model, and the artificial intelligence model is constrained to output structured results in JSON format.
8. An AI-powered intelligent evaluation system for student practical assignments based on a hybrid expert model, characterized in that: include: Input processing module: used to obtain the content of the assignments submitted by students and the task identifier of the teaching task to which they belong; Expected Answer Management Module: Used to store and manage pre-built structured expected answers corresponding to each teaching task, wherein the structured expected answers contain at least dimensional activation information; Gating and expert activation module: used to read the corresponding structured expected answer from the expected answer management module according to the task identifier, and determine the subset of dimension experts to be activated for this evaluation based on the dimension activation information therein; Multi-dimensional expert evaluation module: includes multiple dimension expert units, used to call the corresponding artificial intelligence model according to the instructions of the gating and expert activation module, and generate structured evaluation output for each dimension of the dimension expert subset; The fusion scoring module is used to convert the structured evaluation output of each dimension into dimension scores based on predefined calculation rules, and then perform weighted fusion after normalizing the weights of the expert subsets of the dimensions to obtain the final scoring result. Results output module: Used to output and store structured evaluation results containing the final score and details of each dimension.
9. An electronic device, characterized in that, include: One or more processors; Memory; One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, the one or more computer programs including instructions for performing the method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method as described in any one of claims 1 to 7.