Intelligent subjective question reviewing system and method based on large language model

Abstract

The invention discloses a subjective question intelligent marking system and method based on a large language model, and relates to the technical field of online education and examination intelligent marking, and the system comprises a scoring rule configuration module, an AI scoring engine module, a teacher review interaction module and a result presentation and archiving module. The scoring rule configuration module is used for receiving AI scoring related parameters configured by a user; the AI scoring engine module integrates a large language model and is used for receiving test question related information, examinee answers and AI scoring related parameters and carrying out natural language understanding, knowledge reasoning, logic analysis and standard comparison on the basis of the large language model to obtain an AI scoring result; the teacher re-checking interaction module provides an interaction interface for the user to check the AI scoring result to obtain a final scoring result; and the result presenting and archiving module is used for archiving, storing and visually displaying the final scoring result. According to the invention, the manual reviewing cost can be reduced, and the accuracy and efficiency of subjective question reviewing are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent grading technology for online education and examinations, and in particular to an intelligent grading system and method for subjective questions based on a large language model. Background Technology

[0002] The widespread adoption of online exams has created an urgent need for more efficient and fair grading of subjective questions. Traditional manual grading methods have significant drawbacks: First, grading efficiency is low; teachers often need several days to grade hundreds of papers, failing to meet the requirement of immediate results. Second, labor costs are high; organizing a large number of grading teachers requires not only significant investment in personnel but also corresponding management costs. Third, grading standards are inconsistent; individual differences in the grading criteria and focuses of different teachers lead to inaccurate grading results, affecting the fairness of the exam.

[0003] Most existing automated scoring technologies are based on keyword matching or fixed rules, lacking deep semantic understanding capabilities. They are ill-equipped to handle subjective questions with diverse answer formats and complex logical structures, and cannot accurately grasp the core ideas and logical rigor of candidates' answers, making it difficult to guarantee scoring accuracy.

[0004] Large Language Models (LLMs) have made groundbreaking progress in the field of natural language understanding and generation, possessing powerful semantic analysis, knowledge reasoning, and contextual understanding capabilities. They can deeply interpret the connotation and logical relationships of text, providing reliable technical support for solving the core challenge of automated grading of subjective questions. Therefore, how to provide an intelligent grading system and method for subjective questions based on large language models to reduce the cost of manual grading and improve the accuracy and efficiency of subjective question grading has become a pressing technical problem in this field. Summary of the Invention

[0005] The purpose of this application is to provide an intelligent grading system and method for subjective questions based on a large language model, which can reduce the cost of manual grading and improve the accuracy and efficiency of subjective question grading.

[0006] To achieve the above objectives, this application provides the following solution.

[0007] In the first aspect, this application provides a subjective question intelligent grading system based on a large language model. The subjective question intelligent grading system based on a large language model includes a scoring rule configuration module, an AI scoring engine module, a teacher review and interaction module, and a result presentation and archiving module. The modules work together to realize the automated grading of subjective questions.

[0008] The scoring rule configuration module is used to receive AI scoring-related parameters configured by the user; the AI ​​scoring-related parameters include AI scoring prompt word templates selected or customized by the user, and the AI ​​scoring prompt word templates include: role setting, task definition, rule injection and output constraint related instructions.

[0009] The AI ​​scoring engine module integrates a large language model to receive test question information, candidate answers, and AI scoring parameters. Based on the large language model, it performs natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result. The AI ​​scoring result includes the scoring result and the scoring reason.

[0010] The teacher review interaction module provides an interactive interface for users to review the AI ​​scoring results and obtain the final score. It also supports operations such as modifying scores, regenerating, editing, and discarding scoring reasons.

[0011] The result presentation and archiving module is used to archive, store, and visualize the final scoring results.

[0012] Optionally, the AI ​​scoring prompt word template configuration supports multiple scoring modes, including a scoring mode based on comparison with reference answers and a scoring mode based on scoring criteria.

[0013] Optionally, the AI ​​scoring engine module further includes a preprocessing unit.

[0014] The preprocessing unit is used to perform text cleaning, prompt word construction, and context organization on the received test question information and candidate answers to obtain preprocessing results, and then transmit the preprocessing results to the large language model.

[0015] Optionally, the context organization process follows a preset logical order, groups and marks different types of information, and emphasizes core scoring elements.

[0016] Optionally, the AI ​​scoring engine module further includes an output parsing unit.

[0017] The output parsing unit is used to extract scores, structure the scoring reasons, and evaluate the confidence level of the output results of the large language model.

[0018] Optionally, the confidence assessment metrics include keyword coverage, semantic deviation, and / or score consistency.

[0019] Secondly, this application provides a method for intelligent grading of subjective questions based on a large language model. The method is implemented based on the intelligent grading system for subjective questions based on a large language model as described in any of the first aspects. The method includes the following steps.

[0020] Obtain the AI ​​scoring parameters configured by the user; the AI ​​scoring parameters include AI scoring prompt word templates selected or customized by the user, and the AI ​​scoring prompt word templates include: role setting, task definition, rule injection and output constraint related instructions.

[0021] The system obtains relevant information about the test questions and the candidates' answers. Based on the relevant information about the test questions, the candidates' answers, and the relevant parameters of the AI ​​scoring, it uses a large language model to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result. The AI ​​scoring result includes the scoring result and the reasoning for the scoring.

[0022] The AI ​​scoring results are reviewed to obtain the final score; during the review process, the score can be modified, the scoring reasons can be regenerated, edited, or discarded.

[0023] The final scoring results are archived, stored, and visualized.

[0024] Optionally, the test question information and the candidate's answer are obtained, and based on the test question information, the candidate's answer and the AI ​​scoring parameters, a large language model is used to perform natural language understanding, knowledge reasoning, logical analysis and standard comparison to obtain the AI ​​scoring result, specifically including the following steps.

[0025] Get information about the test questions and the candidates' answers.

[0026] The test questions and the candidates' answers are processed by text cleaning, prompt word construction, and context organization to obtain preprocessed results.

[0027] Based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring results.

[0028] Optionally, based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring results, specifically including the following steps.

[0029] Based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the output results of the large language model.

[0030] The output of the large language model is subjected to score extraction, structuring of scoring reasons, and confidence evaluation to obtain the AI ​​scoring results.

[0031] Optionally, after reviewing the AI ​​scoring results to obtain the final scoring results, the subjective question intelligent grading method based on a large language model further includes a model optimization step, which specifically includes the following steps.

[0032] Based on the AI ​​scoring results and the final scoring results, the user's modification data for the scoring results is determined.

[0033] Based on the modified data, the scoring accuracy and scoring reasoning quality of the large language model are optimized.

[0034] According to the specific embodiments provided in this application, this application has the following technical effects.

[0035] This application provides a subjective question intelligent grading system and method based on a large language model. The system includes a scoring rule configuration module, an AI scoring engine module, a teacher review and interaction module, and a result presentation and archiving module. The scoring rule configuration module receives user-configured AI scoring parameters. The AI ​​scoring engine module integrates a large language model to receive question-related information, student answers, and AI scoring parameters. Based on the large language model, it performs natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result. The teacher review and interaction module provides an interactive interface for users to review the AI ​​scoring result and obtain the final score. The result presentation and archiving module archives and visualizes the final scoring result. Through the collaborative operation of the scoring rule configuration module, AI scoring engine module, teacher review and interaction module, and result presentation and archiving module, automated grading of subjective questions can be achieved. By configuring standardized AI scoring prompts in the scoring rule configuration module and combining this with the efficient reasoning capabilities of the AI ​​scoring engine's large language model, the traditional manual, one-by-one grading method is replaced. This reduces the time required for subjective question grading (which used to take many people several days) to hours or even minutes, significantly meeting the real-time requirements of online exams, reducing manual grading costs, and improving the efficiency of subjective question grading. Furthermore, leveraging the rule injection feature of the AI ​​scoring prompt templates, abstract scoring standards are transformed into unified instructions executable by the model. Through unbiased semantic understanding, logical analysis, and standard comparison using the large language model, problems such as individual evaluation scale differences and fatigue errors inherent in manual grading are completely eliminated. This ensures consistent scoring standards across different candidates and different batches of papers, improving the accuracy of subjective question grading and significantly enhancing the fairness and stability of exam scoring. In addition, the teacher review interaction module provides convenient operations such as score modification and comment regeneration, realizing an efficient collaborative model of "AI initial judgment - human review". This not only reduces the burden on teachers by allowing AI to handle a large amount of repetitive marking work, but also uses human review to make up for possible occasional biases in the model, ensuring the reliability of the scoring results. At the same time, the visualization and structured storage functions of the results presentation and archiving module provide convenience for teachers' teaching analysis and students' review, optimizing the user experience for both teachers and students. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1This is a schematic diagram of the structure of a subjective question intelligent grading system based on a large language model, provided as an embodiment of this application.

[0038] Figure 2 This is a schematic diagram of the core content of AI-based review provided in an embodiment of this application.

[0039] Figure 3 This is a schematic diagram of the AI ​​grading technology architecture provided in one embodiment of this application.

[0040] Figure 4 This is a schematic diagram of an AI-based review process provided in one embodiment of this application.

[0041] Figure 5 This is a flowchart illustrating an intelligent grading method for subjective questions based on a large language model, provided as an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] The purpose of this application is to provide a subjective question intelligent grading system and method based on a large language model, so as to achieve high efficiency, accuracy, automation, fairness and low cost in subjective question grading, while optimizing the user experience for both teachers and students, and solving the problems of low efficiency, high cost, low accuracy and low fairness in traditional grading methods.

[0044] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] This embodiment provides an intelligent grading system for subjective questions based on a large language model, such as... Figure 1 As shown, the intelligent subjective question grading system based on a large language model includes a scoring rule configuration module, an AI scoring engine module, a teacher review and interaction module, and a result presentation and archiving module. These modules work together to achieve automated grading of subjective questions.

[0046] The scoring rule configuration module receives AI scoring-related parameters configured by the user. These parameters include user-selected or user-defined AI scoring prompt templates. The AI ​​scoring prompt templates include instructions related to role setting, task definition, rule injection, and output constraints.

[0047] The AI ​​scoring engine module integrates a large language model to receive test question information, student answers, and AI scoring parameters. Based on the large language model, it performs natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result. The AI ​​scoring result includes a score and a reasoning behind the score.

[0048] The teacher review interaction module provides an interactive interface for users to review the AI ​​scoring results and obtain the final score. It also supports operations such as modifying scores, regenerating, editing, and discarding scoring reasons.

[0049] The result presentation and archiving module is used to archive, store, and visualize the final scoring results.

[0050] As an optional implementation, the AI ​​scoring prompt word template configuration supports multiple scoring modes, including a scoring mode based on comparison with reference answers and a scoring mode based on scoring criteria.

[0051] As an optional implementation, the AI ​​scoring engine module also includes a preprocessing unit.

[0052] The preprocessing unit is used to perform text cleaning, prompt word construction, and context organization on the received test question information and candidate answers to obtain preprocessing results, and then transmit the preprocessing results to the large language model.

[0053] As an optional implementation, the context organization process follows a preset logical order, groups and marks different types of information, and emphasizes core scoring elements.

[0054] As an optional implementation, the AI ​​scoring engine module also includes an output parsing unit.

[0055] The output parsing unit is used to extract scores, structure the scoring reasons, and evaluate the confidence level of the output results of the large language model.

[0056] As an optional implementation, the confidence assessment metrics include keyword coverage, semantic deviation, and / or score consistency.

[0057] This application proposes an intelligent subjective question grading system (hereinafter referred to as the AI ​​grading system) based on a large language model. This system is deeply integrated with an online examination platform in its architecture and mainly includes a scoring rule configuration module, an AI scoring engine module, a teacher review interaction module, and a result presentation and archiving module. Corresponding to each of the above modules, the system's characteristics mainly include scoring rule configuration, AI scoring engine, teacher review interaction, and result presentation and archiving. The system's main functions, scoring modes, prompts, and parameter configurations are as follows: Figure 2 As shown.

[0058] The scoring rule configuration module is used by administrators or teachers to enable the AI ​​scoring function during the exam creation stage, and to select or configure corresponding AI scoring prompt word templates for different subjective question types, including multiple modes such as "scoring based on reference answer comparison" and "scoring based on scoring criteria (such as essays)".

[0059] AI scoring prompt word templates are a structured set of text instructions. Their core function is to transform the scoring rules and intentions of human experts into standardized instructions that large language models can accurately understand and execute. Specifically, their functions include the following:

[0060] (1) Role setting: Clearly instruct the LLM to play the role of "subject marking expert" in this interaction, so that it can activate the knowledge base and evaluation criteria in the relevant field.

[0061] (2) Task definition: Clearly define the scoring task, including input elements (question stem, reference answer, candidate's answer), scoring dimensions (such as content, logic, expression) and output format.

[0062] (3) Rule injection: Transform abstract scoring criteria (such as "logical clarity") into specific requirements that the model can operate, such as "whether the argument chain of the analysis answer is complete and whether there is a clear causal relationship between the sub-arguments".

[0063] (4) Output constraints: Strictly define the structure and format of the model output, for example, require output in a fixed format of "score: X, comment: Y" to facilitate subsequent automated parsing by the system.

[0064] The subsequent AI grading process uses this template through the following steps.

[0065] (1) Template instantiation: When the system receives a candidate's answer to be graded, it uses the question stem, reference answer, candidate's answer and preset scoring rules as variables to fill the corresponding placeholders of the predefined AI scoring prompt word template, forming a complete prompt word containing specific content.

[0066] (2) Prompt input: The complete prompt is passed to the large language model through the application programming interface (API).

[0067] (3) Model reasoning: Based on the prompt words and context, the large language model performs deep semantic understanding, knowledge reasoning and standard comparison on the candidate's answer, and generates a structured output.

[0068] (4) Output of results: The text output by the model strictly follows the format required by the template, thereby ensuring that the system can extract scores and comments stably and accurately.

[0069] The AI ​​scoring engine module, as the core of the system, integrates a large language model. This module receives the question stem, the reference answer (optional), the candidate's answer, and scoring prompts. Through natural language understanding and semantic-syntactic analysis, it intelligently scores the candidate's answer and generates corresponding scoring reasons (AI comments).

[0070] The teacher review module provides a management interface for teachers to review AI-generated scores. Teachers can modify scores, change (regenerate) AI comments, edit or choose not to use comments, achieving efficient collaboration between AI initial assessment and human review.

[0071] The technical implementation process of the "Replace" operation (regenerate) function is as follows.

[0072] (1) When the teacher triggers the “change” operation, the system does not simply repeat the initial scoring request, but constructs a new, more guiding prompt word and calls LLM again.

[0073] (2) Task reiteration: It is reiterated that the operation should be based on the same test questions, reference answers and scoring criteria.

[0074] (3) Result anchoring: clearly indicate the previous rating result (including score and comments) and instruct the LLM to keep the rating score absolutely unchanged.

[0075] (4) Generation requirements: The LLM is required to generate a new evaluation based on the same scoring results, using different analytical perspectives, wording, and expression styles. For example, the model can be instructed to "restate the advantages from the perspective of the depth of knowledge mastery instead of the original perspective of the completeness of the answer" or "rewrite the improvement suggestions section using more encouraging language".

[0076] In this way, the system achieves diversification and personalization of comments while ensuring the objectivity and consistency of the scoring.

[0077] The results presentation and archiving module is used to securely store the final scores and comments and present them to teachers (including AI marking and management functions) and students respectively (the interface is consistent with human marking, so there is no sense of difference).

[0078] In this embodiment, the four core modules—the scoring rule configuration module, the AI ​​scoring engine module, the teacher review and interaction module, and the result presentation and archiving module—achieve the grading process through the following collaborative workflow.

[0079] (1) Initialization configuration (scoring rule configuration module): When creating an exam, teachers use this module to select or define an AI scoring prompt template (e.g., "High School History Essay Question Scoring Template V1") for a specific subjective question and enable the AI ​​scoring function. This configuration information is saved to the system database.

[0080] (2) Triggering and Execution (AI Scoring Engine Module): After the examinee submits the paper, the system automatically combines the question information, the examinee's answer and the stored AI scoring prompt word template, and sends it to the AI ​​scoring engine. The AI ​​scoring calls the LLM to complete the scoring and generates the preliminary score and comments.

[0081] (3) Manual review and correction (teacher review interaction module): Teachers can view the list of AI grading results through the teacher review interaction interface. Teachers can directly modify the score of any result, or perform operations such as "change" or editing on the comments. All modification actions are recorded by the system.

[0082] (4) Results Consolidation and Presentation (Results Presentation and Archiving Module): After the teacher completes the review and confirmation, this module will persistently store the final score and comments, and present them according to the role permissions: the teacher will be shown complete information (including AI initial judgment marking and modification records), while the student will be presented with the final result interface that is no different from manual grading. At the same time, the teacher's modification data is fed back and stored for model optimization.

[0083] The subjective question intelligent grading system proposed in this application, based on a large language model, mainly relies on the following key technologies.

[0084] (1) Large language model: The large language model is the core of the system, which provides powerful semantic understanding, knowledge reasoning and logical analysis capabilities.

[0085] (2) Prompt Engineering: The design of the prompts directly affects the scoring effect of LLM and needs to be optimized according to different question types and scoring rules.

[0086] (3) Semantic understanding: The system needs to be able to accurately understand the meaning of the question stem, the reference answer and the candidate's answer, and identify keywords, key concepts and logical relationships.

[0087] (4) Knowledge reasoning: The system needs to be able to use the knowledge stored in the LLM to reason and expand the candidate's answer, judge whether it is consistent with the question, and explore its potential value.

[0088] In this embodiment, the prompt word template of the prompt word scoring template includes the following content.

[0089] Template A: Basic general template (suitable for most short answer, question answer, and short essay questions).

[0090] Applicable question types: questions with clearly defined knowledge points, such as definition explanations, fact summaries, and list of key points.

[0091] Core logic: Semantically match student answers with the key points of the pre-set reference answers, and score them according to the coverage ratio.

[0092] AI Scoring Prompt Template: You are a strict subject teacher. Please score the student's answer according to the following requirements: Question: {Question}; Reference Answer Core Points (Keywords / Phrase); {Reference Answer Key Point 1}; {Reference Answer Key Point 2}; (List a maximum of 5-7 core points). Scoring Rules: (1) Full marks: {Full marks}. (2) Please evaluate whether the student's answer covers the above core points. The score for each key point is evenly distributed. (3) Coverage Judgment Criteria: The meaning is the same or highly similar, and the words are not required to be completely identical. (4) If there are obvious factual errors or irrelevant content, 1-2 points may be deducted at the discretion of the grader (the total score shall not exceed {Full marks}). Student Answer: {Student Answer} Please output in JSON format; Brief Comments: First summarize the advantages of the answer, and then point out the missing key points or errors. Configuration Items: Reference Answer Core Points, Full Marks, Error Deduction Range.

[0093] Template B: Structure-Content Two-Dimensional Essay Question Template.

[0094] Applicable question types: Opinion argumentation, material analysis, short essays, and other questions that require logical development.

[0095] The core logic is to score based on two independent dimensions: structural logic and content quality, and then calculate the total score.

[0096] AI Scoring Prompt Template: You are a professional examiner. Please evaluate the following essay question answers from two dimensions: “Structure Logic” and “Content Quality”. Question: {Question}; Scoring Dimensions and Standards ({Full Marks}): (1) Structure Logic (40%): Is the viewpoint clear and have clear argument sentences? Are the levels of the argument clear and are the paragraphs connected naturally? Is the conclusion an effective summary or sublimation? (2) Content Quality (60%): Are the relevant concepts and theories accurately used? Are the arguments sufficient, relevant, and able to effectively support the viewpoint? Is the analysis in-depth and does it have unique insights or critical thinking? Student Answer: {Student Answer}, please output in JSON format. Scoring Task: (1) Give the “Structure Logic” score (S_structure) and the “Content Quality” score (S_content) respectively, and calculate the weighted total score: Total Score (2) The total score shall not exceed the maximum score. Dimensional comments: first evaluate the structure, then the content. Specifically point out the strengths and weaknesses. Configuration items: maximum score, structure / content weight ratio, specific scoring criteria for each dimension.

[0097] Template C: Step-by-step calculation problem template.

[0098] Applicable question types: Calculation problems in mathematics, physics, chemistry, etc. that require a solution process.

[0099] Core logic: We value not only the final result but also the correctness of the problem-solving steps. We adopt a "step-by-step points + result points" model.

[0100] AI Scoring Prompt Template: You are a science teacher. Please score the following calculation problem answers step by step. Problem: {Problem}; Standard solution steps and score allocation: Step 1 (describe key step 1): {score}; Step 2 (describe key step 2): {score}; Final correct answer: {result score} (total score = {full score}). Scoring rules: (1) If a step is correct (formula, method, key transformation is correct), the full score of that step will be given. (2) If a step is partially correct or contains a typo, half or a small score of that step will be given. (3) If a step is missing or completely wrong, 0 points will be given. (4) If the final result is correct, the result score will be given; if the result is wrong but the process has correct steps, the process score will still be given. Student answer: {student answer}, please output in JSON format. Overall evaluation: affirm the correct steps, point out the key wrong steps and result problems. Configuration items: detailed step breakdown, score of each step, result score.

[0101] Template D: English essay scoring template.

[0102] Applicable question type: Standardized English essay question type.

[0103] Scoring logic: While ensuring the core evaluation dimensions, priority is given to evaluating efficiency and the inspirational nature of the feedback.

[0104] AI Scoring Prompt Template: You are an English examiner who strictly follows standardized scoring criteria. Please score the essay according to the following dimensions. Essay title: {title}; Writing type: {argumentative essay / graph / letter, etc.}; Word count requirement: {suggested 150-250 words}. Scoring dimensions and weights ({full marks}): (1) Task Response and Content Completion (Task Response) {weight 1}%: Does it fully respond to all parts of the topic? Is the viewpoint clear and the argument sufficient? Are there relevant examples to support it? (2) Coherence and Cohesion (Coherence and Cohesion) {weight 2}%: Is the article structure clear (introduction-body-conclusion)? Are the paragraphs divided reasonably? Are the conjunctions used appropriately and in variety? (3) Lexical Resource (Lexical Resource) {weight 3}%: Is the vocabulary rich and does it avoid repetition? Are the words used accurate and appropriate? Do you try to use more advanced vocabulary? How many spelling errors are there? (4) Grammatical Range and Accuracy (weight 4%): Whether the sentence structure is diverse (simple sentences, compound sentences, and complex sentences are mixed); the frequency and severity of grammatical errors; whether the punctuation is used correctly. Scoring details for each dimension: Excellent (90-100%): fully meets the requirements with only a few minor errors; Good (75-89%): basically meets the requirements with some errors but not affecting understanding; Average (60-74%): partially meets the requirements with obvious errors; Needs improvement (<60%): fails to meet the basic requirements. Student essay: {student answer}. Scoring task: (1) Calculate the scores for the four dimensions respectively: Dimension score = dimension full score × performance percentage; (2) Calculate the total score: Total score = sum of scores for each dimension; (3) Statistically analyze the high-frequency error types in the essay (such as tense errors, subject-verb disagreement, missing conjunctions, etc.). Please output in JSON format. Summary comments: First affirm the advantages, then point out the main problems and directions for improvement.

[0105] This application proposes an intelligent subjective question grading system based on a large language model. The technical architecture aims to simulate the human grading process, employing cue word engineering and leveraging the powerful capabilities of LLM (Language Modeling) for semantic understanding, knowledge reasoning, and logical analysis, thereby achieving objective and efficient subjective question grading. Figure 3 As shown, the technical architecture is mainly divided into four core layers: input layer, preprocessing layer, LLM core processing layer and output parsing layer, each with its specific functions and components.

[0106] The input layer is the entry point for the system to receive information related to the test questions to be graded. It is responsible for receiving the following four types of key information.

[0107] (1) Question stem: A complete description of the question, including the question content, background information and requirements.

[0108] (2) Reference answer: The standard answer provided by the expert is used as the basis for scoring.

[0109] (3) Candidate's answer: The text of the candidate's answer to the test question.

[0110] (4) Scoring rules: detailed scoring criteria and rules, such as keywords, logical points, expression norms, etc.

[0111] The main task of the input layer is to integrate this information and pass it to the next layer for processing.

[0112] The preprocessing layer is responsible for cleaning, transforming, and organizing the information received from the input layer so that the LLM can better understand and process it. This layer contains the following three main components.

[0113] (1) Text cleaning: Remove noise from the text, such as HTML tags, special characters, extra spaces, etc., and convert the text into a uniform encoding format.

[0114] (2) Prompt Construction: Based on the question stem, reference answer, and scoring rules, construct appropriate prompts to guide LLM in correct semantic understanding and reasoning. The design of prompts is crucial, as it directly affects the scoring effect of LLM.

[0115] Prompt word construction is a structured information assembly process, and its specific implementation is as follows.

[0116] The system's pre-built AI-generated scoring prompt template is a text framework containing multiple preset parts. During construction, the system uses string replacement or a template engine to populate the corresponding positions in the framework with specific business data. A typical prompt structure is shown below.

[0117] 1) Role and task: You are a senior subject teacher. Please score the candidates' answers strictly according to the following standards.

[0118] 2) Scoring criteria: The maximum score for this scoring is {maximum score} points, which is mainly considered from the following dimensions.

[0119] Content completeness ({weight percentage}%): {detailed description}.

[0120] Logical rigor ({weighting percentage}%): {detailed description}.

[0121] Standardization of language expression ({weighting percentage}%): {detailed description}.

[0122] 3) Enter information, including the following:

[0123] Title: {Question Stem}

[0124] Suggested answer: {Suggested answer}.

[0125] Candidate's answer: {Candidate's answer}.

[0126] 4) Output requirements, including the following:

[0127] Please strictly adhere to the following output format and do not add any other content.

[0128] Fraction: [Enter the fraction here, rounded to one decimal place].

[0129] Commentary: [This section discusses the advantages and disadvantages from three perspectives and provides suggestions for improvement].

[0130] (3) Context organization: Organize the question stem, reference answer, candidate answer and scoring rules into a context according to a certain logical order so that LLM can fully understand the questions and answers and make accurate comparisons and evaluations.

[0131] In this embodiment, context organization aims to build a logically clear and informationally complete "thinking scenario" for LLM, and its fusion process follows the following principles.

[0132] Logical order: Information is organized in the order of "problem definition -> evaluation criteria -> reference basis -> object to be evaluated". This conforms to the logical habits of human problem analysis and helps the model understand.

[0133] Information grouping and labeling: Use clear chapter titles (such as title, standard, reference, answer to be evaluated) to block information of different natures to prevent model confusion.

[0134] Emphasis on key information: The core elements of the scoring criteria, such as weights, key scoring points, and veto items, are emphasized using special symbols (such as bolding), numbering, or as separate paragraphs to guide the model to give them sufficient attention.

[0135] Format standardization: Ensure that all input text is cleaned, irrelevant formatting symbols are removed, and uniform encoding (such as "UTF-8") is used to guarantee the accuracy of information transmission.

[0136] The goal of the preprocessing layer is to provide high-quality input for the LLM, ensuring that it can be effectively processed in subsequent steps.

[0137] The LLM core processing layer is the heart of the entire system. It leverages the powerful capabilities of LLM for semantic understanding, knowledge reasoning, logical analysis, and standard comparison.

[0138] The LLM core processing layer achieves the following functions through its trillions of internal parameters, and its technical essence can be described as follows.

[0139] Semantic understanding: This involves understanding the meaning of test question stems, reference answers, and student answers, and identifying keywords, key concepts, and logical relationships. The model first converts the input text (question stems, answers, etc.) into semantic vectors (word embedding vector sequences) in a high-dimensional space. Through a self-attention mechanism, it calculates the correlation strength between vectors, thereby understanding the deeper meaning of words, phrases, and sentences in the text and the contextual relationships between them.

[0140] Knowledge Reasoning: Utilizing the knowledge stored internally by the LLM (Learning Model), the model reasones and expands upon test-takers' answers, determining their relevance to the question and uncovering their potential value. The model uses knowledge learned from massive amounts of corpus during training to form a parameterized knowledge base. When processing answers, it activates relevant knowledge neurons to perform common-sense judgments, causal inferences, and logical deductions. For example, it determines whether a statement about a historical event contains a correct causal chain.

[0141] Logical Analysis: This involves analyzing the logical structure and argumentation process of the candidate's answer to determine its rigor and rationality, and identifying any logical flaws. The model constructs a logical structure diagram of the answer by analyzing the text's dependency syntax and semantic roles, assessing the completeness of its argumentation chain, the validity of its premises and conclusions, and the presence of logical fallacies.

[0142] Standard Comparison: The candidate's answer is compared with the reference answer to determine whether it covers the key points, and a score is assigned according to the scoring rules. This process can be quantified as a multi-dimensional weighted scoring function. The model calculates the similarity between the semantic vector of the candidate's answer and the semantic vector of the reference answer in a multi-dimensional space, and then performs a comprehensive weighted calculation based on the scoring rules.

[0143] The goal of the LLM core processing layer is to simulate the thought process of a human rater and to provide a comprehensive and in-depth evaluation of the candidate's answers.

[0144] The raw output of the LLM core processing layer is an unstructured block of plain text that strictly follows prompt word instructions.

[0145] For example: Score: 8.5.

[0146] Comments: Strengths: The answer covers the core knowledge points and has a clear logical structure. Weaknesses: The second point lacks depth in its discussion. Suggestion: Further analysis could be developed by incorporating specific historical facts.

[0147] The parsing and generation process is as follows.

[0148] Rule-based parsing: The system uses regular expression-based pattern matching rules to extract key information from the above text.

[0149] Structured storage: The parsed scores are converted into floating-point numbers for storage, while the comment text is stored as a complete string.

[0150] Post-processing of comments: The system can further standardize and reorganize the captured comment text according to a predefined framework (such as a three-part structure of "strengths-weaknesses-suggestions") to ensure that all comments are structurally consistent and improve user experience. This process does not change the substantive content of the comments, only optimizes the format.

[0151] The output parsing layer is responsible for parsing, extracting, and transforming the LLM output to generate highly readable ratings and comments. The output parsing layer comprises the following three main components.

[0152] (1) Score extraction: Extract the score from the LLM output and convert it into a standard numerical format.

[0153] (2) Comment generation: Based on the analysis results of LLM, comments are automatically generated for the candidates' answers, pointing out their strengths and weaknesses and providing suggestions for improvement.

[0154] (3) Confidence assessment: assess the confidence of the LLM scoring results and provide corresponding confidence indicators, such as scoring consistency and keyword coverage.

[0155] The goal of the output parsing layer is to transform the LLM scoring results into information that is easy to understand and use, and to provide a reference for users.

[0156] This application achieves true semantic-level scoring through deep semantic understanding, surpassing keyword matching. It dynamically adjusts scoring strategies based on different question types and scoring criteria. Furthermore, it provides explanatory output, offering detailed scoring justification and improvement suggestions. Human feedback is continuously used to optimize scoring accuracy and comment quality. Additionally, it supports high-concurrency batch processing for instant score generation, ensuring that AI-based grading maintains high efficiency while achieving accuracy and educational value comparable to human grading.

[0157] The following describes the operation process and intelligent grading workflow of the subjective question intelligent grading system based on a large language model proposed in this application. Figure 4 As shown, it includes the following steps.

[0158] (1) Examination configuration, including the following:

[0159] 1) The administrator enables the AI ​​scoring function globally in the exam management backend.

[0160] 2) When creating or editing exams, teachers can check the "AI scoring" option and, if necessary, check "Show AI comments to students".

[0161] 3) In question type management, assign appropriate AI scoring prompt word templates to the required subjective question types (such as short answer questions, essay questions, composition questions, etc.).

[0162] (2) Students submit their papers, including the following content.

[0163] 1) Students answer questions and complete the exam on a PC webpage. For subjective questions, answers must be entered in plain text format.

[0164] 2) After students submit their exam papers, the exam system immediately sends the scoring task, which includes the question stem, reference answer (if any), and the student's answer, to the AI ​​scoring queue.

[0165] (3) AI scoring, including the following:

[0166] 1) The AI ​​scoring engine retrieves scoring tasks from the queue.

[0167] 2) The engine combines the scoring instructions, question information, and answer content according to the preset AI scoring prompt word template, and calls the large language model API.

[0168] 3) The large language model understands and analyzes the candidates' answers, and outputs the scoring results and detailed scoring reasons.

[0169] 4) The system will link the scores and comments returned by the AI ​​with the candidates' answer sheets.

[0170] (4) Teacher review, including the following:

[0171] 1) Teachers can log in to the grading management interface to view all student answer sheets that have been graded by AI.

[0172] 2) Teachers can quickly view the scores and comments provided by the AI. If they believe the scoring results are inappropriate, they can perform the following actions.

[0173] ① Modify score: Directly adjust the score.

[0174] ② Optimize comments: Click "Change" to let AI regenerate comments, or manually edit the comments.

[0175] 3) After confirming the marking results, the teacher clicks "Complete Marking".

[0176] (5) Publish the results, including the following:

[0177] 1) Once the entire grading process is completed, the system will release the exam results.

[0178] 2) Students log in to the system to view their exam papers, where they can see their scores and comments (if configured to be displayed). The student interface is identical to that of manual grading, ensuring a consistent exam experience.

[0179] Based on the same inventive concept, this application also provides a method for intelligent grading of subjective questions based on a large language model, which is based on the aforementioned intelligent grading system for subjective questions based on a large language model. The solution provided by this method is similar to the implementation scheme described in the aforementioned system. Therefore, the specific limitations in the embodiments of the method for intelligent grading of subjective questions based on a large language model provided below can be found in the limitations of the intelligent grading system for subjective questions based on a large language model described above, and will not be repeated here.

[0180] In one exemplary embodiment, such as Figure 5 As shown, a method for intelligent grading of subjective questions based on a large language model is provided, including the following steps.

[0181] S1: Obtain the AI ​​scoring parameters configured by the user. These AI scoring parameters include user-selected or user-defined AI scoring prompt templates, which include instructions related to role settings, task definitions, rule injection, and output constraints.

[0182] S2: Obtain test question information and candidate answers, and based on the test question information, candidate answers, and AI scoring parameters, use a large language model to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result. The AI ​​scoring result includes the scoring result and the reasoning behind the scoring.

[0183] S3: Review the AI ​​scoring results to obtain the final scoring result; during the review, the score can be modified, the scoring reasons can be regenerated, edited, and discarded.

[0184] S4: Archive, store, and visualize the final scoring results.

[0185] As an optional implementation, step S2 obtains test question information and candidate answers, and based on the test question information, candidate answers, and AI scoring parameters, uses a large language model to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result, specifically including the following steps.

[0186] S21: Obtain information related to the test questions and the candidates' answers.

[0187] S22: Perform text cleaning, prompt word construction, and context organization on the test question-related information and the candidate's answer to obtain the preprocessed result.

[0188] S23: Based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring results.

[0189] As an optional implementation, step S23, based on the preprocessing results and the AI ​​scoring parameters, calls the large language model to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring results, specifically including the following steps.

[0190] S231: Based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis and standard comparison to obtain the output results of the large language model.

[0191] S232: Extract scores, structure scoring reasons, and evaluate confidence levels from the output of the large language model to obtain AI scoring results.

[0192] As an optional implementation, after reviewing the AI ​​scoring results in step S3 to obtain the final scoring results, the subjective question intelligent grading method based on a large language model further includes a model optimization step, which specifically includes the following steps.

[0193] A1: Based on the AI ​​scoring result and the final scoring result, determine the user's modification data for the scoring result.

[0194] A2: Based on the modified data, optimize the scoring accuracy and the quality of the scoring reasoning of the large language model.

[0195] The subjective question intelligent grading system and method based on a large language model proposed in this application have the following advantages.

[0196] (1) Extremely improved grading efficiency and instant scoring: By pre-setting standardized prompt word templates through the scoring rule configuration module, combined with the efficient reasoning ability of the large language model of the AI ​​scoring engine module, the traditional manual grading mode is replaced. The subjective question grading work that originally required many people and days to complete is shortened to hours or even minutes. It supports the rapid scoring of tens of thousands of test papers after the exam, greatly meeting the "instant" requirements of online exams. The grading efficiency is more than 100 times higher than that of manual grading.

[0197] (2) Unified and standardized scoring criteria to ensure exam fairness: By leveraging the "rule injection" feature of AI scoring prompt word templates, abstract scoring criteria are transformed into unified instructions that the model can execute. Through large language models, unbiased semantic understanding, logical analysis and standard comparison are performed to completely eliminate problems such as individual evaluation scale differences and fatigue errors in manual grading, ensuring that the scoring criteria of different candidates and different batches of test papers are consistent, and significantly improving the fairness and stability of exam scoring.

[0198] (3) Significantly reduced human and management costs: The system automates the core marking process of subjective questions, requiring only a small number of teachers to review the results. It does not require a large-scale marking team, which can reduce marking human costs by at least 80%. At the same time, it reduces investment in management aspects such as marking organization and coordination, standard training, and dispute arbitration, further compressing the overall examination operation costs.

[0199] (4) The scoring is both accurate and explanatory: Relying on the deep semantic understanding and knowledge reasoning ability of the large language model, it breaks through the limitations of traditional keyword matching and can accurately grasp the core viewpoints, logical structure and knowledge application level of the candidates' answers. The scoring accuracy is comparable to that of professional teachers. At the same time, it generates structured scoring reasons, clearly presents the scoring basis, advantages and disadvantages, and provides targeted feedback to candidates, combining scoring accuracy and educational guidance.

[0200] (5) Flexible and adaptable to diverse marking scenarios with strong scalability: The scoring rule configuration module supports users to select or customize prompt word templates, which can be adapted to various subjective question types such as short answer questions, essay questions, and composition questions, as well as the scoring requirements of different subjects and different difficulty levels; at the same time, the scoring logic can be flexibly adjusted through instructions such as role setting and task definition to meet personalized marking scenarios, and has strong scenario adaptability and scalability.

[0201] (6) Human-computer collaboration optimizes the experience, balancing efficiency and reliability: The teacher review interaction module provides convenient operations such as score modification and comment regeneration, realizing an efficient collaborative mode of "AI initial judgment - human review". It reduces the burden on teachers by having AI undertake a large amount of repetitive marking work, and makes up for possible occasional deviations of the model by human review, ensuring the reliability of the scoring results. At the same time, the visualization and structured storage functions of the result presentation and archiving module provide convenience for teachers' teaching analysis and students' review, optimizing the user experience for both teachers and students.

[0202] (7) The results management is standardized and efficient, ensuring data security: The results presentation and archiving module systematically archives and stores the final scoring results, and supports differentiated display according to role permissions (teachers can view the complete review process, and candidates can view the final results), which not only ensures the standardization of data management, but also maintains the seriousness of the examination results; at the same time, the structured storage facilitates subsequent data statistics, traceability and reuse, and provides data support for teaching quality analysis.

[0203] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A subjective question intelligent grading system based on a large language model, characterized in that, The intelligent subjective question grading system based on a large language model includes a scoring rule configuration module, an AI scoring engine module, a teacher review and interaction module, and a result presentation and archiving module. The modules work together to achieve automated grading of subjective questions. The scoring rule configuration module is used to receive AI scoring-related parameters configured by the user; the AI ​​scoring-related parameters include AI scoring prompt word templates selected or customized by the user, and the AI ​​scoring prompt word templates include: role setting, task definition, rule injection and output constraint related instructions; The AI ​​scoring engine module integrates a large language model to receive test question information, candidate answers, and AI scoring parameters. Based on the large language model, it performs natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result. The AI ​​scoring result includes the score and the reasoning behind the score. The teacher review interaction module provides an interactive interface for users to review the AI ​​scoring results and obtain the final score. It also supports operations such as modifying scores, regenerating, editing, and discarding scoring reasons. The result presentation and archiving module is used to archive, store, and visualize the final scoring results.

2. The intelligent grading system for subjective questions based on a large language model according to claim 1, characterized in that, The AI ​​scoring prompt word template configuration supports multiple scoring modes, including a scoring mode based on comparison with the reference answer and a scoring mode based on the scoring criteria.

3. The intelligent grading system for subjective questions based on a large language model according to claim 1, characterized in that, The AI ​​scoring engine module also includes a preprocessing unit; The preprocessing unit is used to perform text cleaning, prompt word construction, and context organization on the received test question information and candidate answers to obtain preprocessing results, and then transmit the preprocessing results to the large language model.

4. The intelligent grading system for subjective questions based on a large language model according to claim 3, characterized in that, The context organization process follows a preset logical order, groups and marks different types of information, and emphasizes core scoring elements.

5. The intelligent grading system for subjective questions based on a large language model according to claim 3, characterized in that, The AI ​​scoring engine module also includes an output parsing unit; The output parsing unit is used to extract scores, structure the scoring reasons, and evaluate the confidence level of the output results of the large language model.

6. The intelligent grading system for subjective questions based on a large language model according to claim 5, characterized in that, The confidence assessment metrics include keyword coverage, semantic deviation, and / or score consistency.

7. A method for intelligent grading of subjective questions based on a large language model, characterized in that, The intelligent grading method for subjective questions based on a large language model is implemented based on the intelligent grading system for subjective questions based on a large language model as described in any one of claims 1-6. The intelligent grading method for subjective questions based on a large language model includes: Obtain the AI ​​scoring parameters configured by the user; the AI ​​scoring parameters include AI scoring prompt word templates selected or customized by the user, and the AI ​​scoring prompt word templates include: role setting, task definition, rule injection and output constraint related instructions; The system acquires test question information and candidate answers, and based on the test question information, candidate answers, and AI scoring parameters, uses a large language model to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result; the AI ​​scoring result includes the scoring result and the scoring reason. The AI ​​scoring results are reviewed to obtain the final score; during the review process, the score can be modified, the scoring reasons can be regenerated, edited, or discarded. The final scoring results are archived, stored, and visualized.

8. The intelligent grading method for subjective questions based on a large language model according to claim 7, characterized in that, The system acquires test question information and candidate answers, and based on the test question information, candidate answers, and AI scoring parameters, utilizes a large language model to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring result, specifically including: Obtain information related to the test questions and the candidates' answers; The test question-related information and the candidate's answer are subjected to text cleaning, prompt word construction, and context organization processing to obtain preprocessed results; Based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring results.

9. The intelligent grading method for subjective questions based on a large language model according to claim 8, characterized in that, Based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis, and standard comparison to obtain the AI ​​scoring results, specifically including: Based on the preprocessing results and the AI ​​scoring parameters, the large language model is invoked to perform natural language understanding, knowledge reasoning, logical analysis and standard comparison to obtain the output results of the large language model. The output of the large language model is subjected to score extraction, structuring of scoring reasons, and confidence evaluation to obtain the AI ​​scoring results.

10. The intelligent grading method for subjective questions based on a large language model according to claim 7, characterized in that, After reviewing the AI ​​scoring results to obtain the final score, the intelligent grading method for subjective questions based on a large language model further includes a model optimization step, which specifically includes: Based on the AI ​​scoring results and the final scoring results, determine the user's modification data for the scoring results; Based on the modified data, the scoring accuracy and scoring reasoning quality of the large language model are optimized.

Images