Automatic checking method for declared examination based on large model

Abstract

The invention relates to the technical field of mechanical learning simulation training, in particular to an automatic correction method for a declaration examination based on a large model, which comprises the following steps of: firstly, training a general large language model by using a supervision fine tuning method to obtain a vertical field large model suitable for the declaration correction field, and then inputting student answers, question materials and correction specifications; the method comprises the following steps: dividing score points and keyword nodes by using a vertical domain large model, calculating a preliminary score according to the matching degree of the keyword nodes, generating key sub-node comments and improvement suggestions, performing de-duplication processing on the score points, finally summarizing scores and comments, and outputting a final total score and personalized feedback; according to the method, the large model in the vertical field is trained through supervision and fine adjustment, so that the large model accurately masters professional specifications of application and correction, score points with similar semantics but different expressions can be recognized, the problems that a traditional rule system is rigid and the adaptability of a general large model field is poor are effectively solved, and the misjudgment rate is remarkably reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning simulation training, and in particular to an automatic correction method for essay examinations based on a large model. Background Art

[0002] The essay test is an important part of the civil service and public institution examinations. As the basic question type of the essay test, it mainly tests the examinee's ability to extract key information from complex materials and make high-level generalizations. Due to the high subjectivity of the essay test, the grading process relies on manual scoring, which has problems such as low efficiency, inconsistent scoring standards and untimely feedback. With the rapid development of artificial intelligence technology, especially the widespread application of large language models (LLMs), automated grading systems have become a potential solution to improve the efficiency and consistency of essay test scoring.

[0003] In the existing technology, some essay grading systems rely on preset rules and templates to score candidates' answers through methods such as keyword matching and sentence analysis. For example, the system will determine the scoring points based on whether the answer contains specific keywords or phrases (such as "broaden the channels for public opinion"). This method is common in early educational technology platforms, such as the essay scoring function of some online learning systems. This method can only recognize preset keywords or sentence patterns and cannot handle answers with similar semantics but different expressions, resulting in insufficient scoring accuracy. For example, if a candidate uses "expand the channels for communicating public opinion" instead of "broaden the channels for public opinion", it may be mistakenly judged as not scoring, and no detailed comments and personalized suggestions can be provided. It can only point out the missing keywords, which makes it difficult to help candidates improve. In addition, the rules of this method need to be manually adjusted for each question type, which makes it difficult to cope with the diversity of materials and questions in the essay test.

[0004] With the development of machine learning and deep learning, some systems utilize large, general-purpose language models (such as BERT and the GPT series) for text analysis and scoring. These systems train models to identify semantic features in answers and then assign scores based on scoring criteria. For example, essay grading tools developed by some education technology companies use semantic similarity calculations and feature extraction to determine whether answers meet scoring requirements. Furthermore, some systems attempt to fine-tune models with a small amount of annotated data to adapt to specific tasks. However, these general-purpose models, without targeted training, struggle to accurately understand the grading standards and specific requirements of essay exams. For example, they may not accurately distinguish between the applicable scenarios for keywords such as "administrative management" and "cultural ecology." Furthermore, this method requires a large amount of annotated data for fine-tuning, resulting in high data acquisition and annotation costs. While these systems can generate scores, the resulting comments are often overly general and lack detailed recommendations for specific sub-nodes. For example, the system may simply indicate that the answer is "incomplete" without specifying the specific missing key points. Furthermore, they fail to strictly deduplicate keyword nodes, potentially resulting in duplicate calculations of the same score, affecting scoring accuracy.

[0005] To sum up, traditional grading methods are inefficient. At the same time, different scorers may have deviations in their understanding and scoring standards of the same answer, resulting in inconsistent scoring results. In addition, existing technologies find it difficult to balance semantic diversity and domain specificity when dealing with essay questions. The rule system is too rigid, and the general large model lacks in-depth optimization for essay grading standards. The feedback generated by existing systems often remains superficial and it is difficult to provide specific suggestions for each scoring point. Existing technologies do not fully consider the deduplication of scoring points, which may lead to repeated calculation of scores. While pursuing efficiency, existing technologies sacrifice personalized feedback capabilities.

[0006] Therefore, in response to the above problems, the present invention proposes an automatic grading method for essay examinations based on a large model. Through SFT fine-tuning, keyword node splitting, comment generation and post-processing steps, the present invention can provide accurate and personalized feedback while ensuring scoring consistency and efficiency, thereby significantly improving the quality and educational value of essay examination grading. Summary of the Invention

[0007] In order to overcome the problems of high misjudgment rate and poor consistency of scoring in essay writing examination grading technology, the present invention proposes an automatic grading method for essay writing examination based on a large model.

[0008] The technical solution of the present invention is: a method for automatically correcting application essays based on a large model, comprising the following steps:

[0009] S1, using supervised fine-tuning methods to train a general large language model to obtain a vertical large model suitable for the application essay correction field;

[0010] S2: Input student answers, question materials, and grading standards, and use the vertical domain model to split scoring points and keyword nodes;

[0011] S3, calculate the preliminary score based on the matching degree of the keyword node;

[0012] S4, generate key sub-node comments and improvement suggestions;

[0013] S5, performing deduplication processing on the scoring points;

[0014] S6, summarizes the scores and comments, and outputs the final total score and personalized feedback.

[0015] Preferably, the training data for supervised fine-tuning includes historical question materials, student answers, grading specifications and corresponding scoring results, and the professionalism of the model for essay grading tasks is optimized by annotating data.

[0016] Preferably, the step of splitting the scoring points and keyword nodes includes splitting the student's answers into multiple scoring points according to the grading specifications, each scoring point corresponds to a scoring item, and extracting the keyword nodes of each scoring point, judging whether the keyword nodes match the requirements of the grading specifications through the vertical field big model, and calculating the preliminary score based on the degree of matching.

[0017] Preferably, the step of generating key sub-node comments and improvement suggestions includes analyzing the core information of each scoring point, evaluating its completeness and accuracy, and generating targeted comments and actionable improvement suggestions based on the correction specifications.

[0018] Preferably, the deduplication process includes identifying and merging synonymous expressions of the same scoring point, so that each key point is scored only once.

[0019] Preferably, the vertical field large model is constructed based on the LLAMA or GPT series models, and is fine-tuned to adapt to the specific needs of essay marking, including optimizing the model's semantic understanding of government documents and policy terms.

[0020] Preferably, the method further comprises a step of verifying the score calculation logic, verifying the correctness of the score of each sub-node through a post-processing module, and checking whether there are omissions or logical errors.

[0021] Preferably, the personalized feedback includes a total score, an overall comment, and specific improvement suggestions for each scoring point. The overall comment summarizes the advantages and disadvantages of the answer, while the sub-node suggestions provide detailed guidance, forming a multi-level feedback system.

[0022] Preferably, the method supports rapid grading of large-scale examinations, with the processing time for a single answer being in seconds.

[0023] Preferably, the method can be extended to other subjective question grading scenarios, and can be adapted to different question types by adjusting training data and grading specifications, and its core process is universal.

[0024] Beneficial effects of the present invention:

[0025] 1. This invention trains a large vertical domain model through supervised fine-tuning (SFT), enabling it to accurately grasp the professional standards of essay writing grading and identify scoring points with similar semantics but different expressions. It effectively solves the problems of rigid traditional rule systems and poor domain adaptability of general large models, making the scoring results closer to the level of human experts and significantly reducing the misjudgment rate.

[0026] 2. Through the key sub-node comment generation technology, the system can provide specific comments and improvement suggestions for each scoring point. Compared with traditional grading that only gives an overall score or a general evaluation, this invention can provide candidates with actionable learning guidance and significantly improve preparation efficiency.

[0027] 3. This invention introduces a scoring point deduplication mechanism, which automatically identifies and merges scoring points with synonymous expressions to avoid repeated scoring of the same key point, solves the problem of inflated scores in the existing technology, and makes the scoring results more credible.

[0028] 4. By reducing the time it takes to mark a single answer from 5-10 minutes for manual review to seconds, while achieving an exponential improvement in marking efficiency through automated processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 What is shown is a schematic diagram of the workflow of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0031] The present invention provides an embodiment: a method for automatically correcting essay examinations based on a large model, comprising the following steps:

[0032] S1, using supervised fine-tuning methods to train a general large language model to obtain a vertical large model suitable for the application essay correction field;

[0033] S2: Input student answers, question materials, and grading standards, and use the vertical domain model to split scoring points and keyword nodes;

[0034] S3, calculate the preliminary score based on the matching degree of the keyword node;

[0035] S4, generate key sub-node comments and improvement suggestions;

[0036] S5, performing deduplication processing on the scoring points;

[0037] S6, summarizes the scores and comments, and outputs the final total score and personalized feedback.

[0038] Furthermore, the present invention first uses supervised fine-tuning (SFT) to train a large vertical field model specifically for essay writing grading, so that it has the ability to accurately understand the grading standards and the semantics of the examinee's answers. Then, through the scoring point splitting and keyword node extraction technology, the student answers are structured and the preliminary scores are calculated. Combined with the key sub-node comment generation module, targeted feedback is provided for each scoring point. Finally, after the scoring points are deduplicated and the scores are verified, the fairness and accuracy of the scoring are ensured. Finally, a complete grading result including the total score, overall comments and improvement suggestions is output. This method not only greatly improves the grading efficiency, but also provides scoring quality and teaching guidance value close to expert level. The present invention also realizes the full process intelligence of essay writing test grading.

[0039] Furthermore, the present invention uses a labeled data set containing historical question materials, student answers, correction specifications and scoring results for supervised fine-tuning (SFT), so that the general large language model can quickly master the professional knowledge of essay correction, including core capabilities such as scoring rules, keyword recognition standards and semantic similarity judgment. This targeted training significantly improves the adaptability of the model in the field of essay correction, ensuring that the correction results not only meet the requirements of the examination specifications, but also can accurately understand the diverse expressions of the candidates' answers, solving the problem of poor application effect of general models in professional fields.

[0040] The step of splitting the scoring points and keyword nodes includes splitting the student's answers into multiple scoring points according to the grading specifications, each scoring point corresponds to a scoring item, and extracting the keyword nodes of each scoring point, judging whether the keyword nodes match the requirements of the grading specifications through the vertical field big model, and calculating the preliminary score based on the degree of matching.

[0041] Furthermore, the scoring point and keyword node splitting technology proposed in the present invention can intelligently decompose student answers into multiple scoring units according to the grading standards, and accurately extract the core keyword nodes of each scoring point, and evaluate the preliminary score by calculating the degree of matching between these keyword nodes and the standard answer. This method not only retains the standardization advantages of the rule system, but also incorporates the semantic understanding ability of the large model, making the scoring process more scientific and reasonable, thereby effectively balancing the rigidity of the scoring standards and the flexibility of semantic understanding.

[0042] Furthermore, the key sub-node comment generation technology of the present invention automatically generates targeted comments and improvement suggestions by deeply analyzing the content quality, completeness and accuracy of each scoring point. These feedbacks not only point out specific problems (such as "insufficient case support"), but also provide actionable improvement plans (such as "suggestions for supplementing local grassroots governance examples in 2023"), forming a personalized and refined guidance system, greatly enhancing the teaching value of the correction results, and helping candidates to accurately improve their writing skills.

[0043] Furthermore, the scoring point deduplication processing mechanism of the present invention identifies and merges different expressions of the same scoring point through semantic analysis (such as treating "strengthening supervision" and "strengthening supervision management" as the same scoring point), thereby ensuring that each core point is scored only once. This processing effectively prevents candidates from obtaining inappropriate scores through synonymous repetition, thereby significantly improving the fairness and accuracy of the scoring results, and solving the problem of duplicate scoring that is prevalent in the existing technology.

[0044] Furthermore, the vertical field grading system constructed based on advanced large models such as LLAMA or GPT in the present invention is particularly good at processing professional content such as government documents and policy terminology through targeted fine-tuning. While maintaining the powerful semantic understanding ability of the general model, it also significantly improves its adaptability to the specific needs of the application test. This professional transformation has enabled the system to reach a level close to that of human experts in terms of correction accuracy and feedback quality.

[0045] Furthermore, the score calculation and verification mechanism designed in the present invention performs logical verification on the score of each sub-node in the post-processing stage to check whether there are any missing scoring points or calculation errors. This quality control measure ensures that the final scoring results strictly comply with the grading specifications, thereby effectively avoiding technical deviations and making the scores output by the system highly reliable and credible.

[0046] The personalized feedback includes the total score, overall comments and specific improvement suggestions for each scoring point. The overall comments summarize the advantages and disadvantages of the answer, while the sub-node suggestions provide detailed guidance, forming a multi-level feedback system.

[0047] The invention achieves a grading speed of seconds, compressing the traditional manual review process, which takes several minutes, to seconds through an automated process. This high efficiency enables it to easily cope with the massive grading needs of large-scale examinations, and can save more than 80% of labor costs while ensuring the quality of grading.

[0048] The method can be extended to other subjective question grading scenarios. By adjusting the training data and grading standards to adapt to different question types, its core process is universal.

[0049] See also Figure 1, further, the workflow of the present invention is described in detail, specifically:

[0050] Based on the professional needs of the application essay correction field, we first collected history test questions, standard answers, scoring rules and expert correction samples as training data sets, and used supervised fine-tuning (SFT) technology to perform domain adaptation training on the pre-trained large language model (such as LLAMA or GPT series), focusing on optimizing the model's understanding of policy terminology, official document sentence patterns, and scoring standards. Through multiple rounds of iterative training, the model mastered the professional knowledge of application essay correction, and finally obtained a vertical large model with domain expertise. This model can accurately understand the correction specifications and identify the key scoring points in the candidates' answers.

[0051] The system receives student answers to be graded, corresponding test materials, and detailed grading specifications. It first performs standardized preprocessing on the input text, including format unification, typo correction, and segmentation processing. It then associates and encodes the test materials, grading specifications, and student answers to construct structured input data, providing a standardized data foundation for subsequent intelligent grading and ensuring that answers in different formats can be correctly processed by the system.

[0052] The vertical field big model uses a combination of deep learning and rules to automatically split student answers into multiple scoring units according to the requirements of the grading specifications. Each scoring unit corresponds to a scoring point. At the same time, it identifies and extracts the core keyword nodes in each scoring unit (such as "improving the regulatory mechanism", "promoting digital transformation", etc.), and then establishes a mapping relationship between the answer content and the scoring criteria, laying the foundation for accurate scoring.

[0053] Based on the extracted keyword nodes, the system calculates their semantic similarity with the key points of the standard answer, and adopts a multi-dimensional scoring algorithm to comprehensively consider factors such as content completeness, expression accuracy, and argument depth to generate a preliminary score for each scoring point. At the same time, it records the scoring basis and judgment reasons to form a traceable scoring process, thereby ensuring that the scoring results are objective and transparent.

[0054] For the scoring results of each scoring point, the system calls on the natural language generation capability of the vertical large model, and combines the correction standards and typical comment templates to automatically generate personalized comments containing specific problem diagnosis and improvement suggestions. The content of the comments not only points out the existing deficiencies (such as "the feasibility analysis of countermeasures is not in-depth enough"), but also provides constructive modification suggestions (such as "it is recommended to supplement specific implementation steps and time nodes"), thereby forming feedback content with teaching value.

[0055] Through semantic similarity calculation and keyword association analysis, the system automatically identifies and merges different expressions of the same scoring point (such as "optimizing approval processes" and "simplifying procedures"), ensuring that each core point is scored only once, thus avoiding the problem of inflated scores caused by repeated scoring. At the same time, the merged scoring points are recalibrated to ensure the fairness and accuracy of the scoring results.

[0056] The system calculates the final score of the answer based on the cumulative scores of the scoring points after deduplication, and performs logical verification on extreme scores or abnormal situations to ensure that the calculation of the total score meets the requirements of the grading standards. At the same time, it generates summary comments reflecting the overall answer level and evaluates the quality of the answer from a macro perspective.

[0057] The grading results are output in a structured form, including detailed scores for each scoring point, reasons for deductions, improvement suggestions, final total scores and overall evaluation. The grading results support multiple output formats, making it convenient for candidates to review and understand. At the same time, the system will automatically record the complete grading process data, thus providing a basis for subsequent quality review and model optimization.

[0058] The system regularly collects newly generated grading data and expert review opinions, updates model parameters through incremental learning, and continuously optimizes the accuracy of grading and the quality of comments. At the same time, it dynamically adjusts grading standards based on changes in question types and policy updates, ensuring that the system always maintains optimal grading performance and forms a benign self-evolution mechanism.

[0059] Furthermore, the present invention provides an embodiment for basic application scenarios:

[0060] When grading the provincial civil service examination essay, the system receives candidates' answers on "grassroots governance innovation". First, it uses the pre-trained vertical field large model to identify scoring points such as "grid management", "digital empowerment", and "mass participation mechanism". Then, it scores the matching degree of each point according to the scoring standards of the provincial examination. In response to the problem of "insufficient feasibility of countermeasures", it generates specific improvement suggestions such as "Suggestions for supplementing grid governance cases in the Yangtze River Delta region". Finally, it completes the grading report within 3 seconds, including the score, detailed score of each point and 5 targeted suggestions. The examiner's random review confirmed that the scoring accuracy reached 96%.

[0061] Furthermore, the present invention provides an embodiment for processing complex expressions:

[0062] Faced with the diverse expressions used by a candidate in answering the topic of "rural revitalization" (such as expressing "industrial integration" as "linked development of the three industries" and "coordinated promotion of characteristic agriculture and rural tourism", etc.), the system accurately identified that these expressions all pointed to the same scoring point, and then merged the scores through semantic clustering technology to avoid repeated scoring. At the same time, in response to the problem of "unclear policy implementation path", it generated professional guidance such as "it is recommended to supplement the implementation path according to the framework of 'enterprise-led-cooperative organization-farmer participation'", demonstrating excellent semantic understanding and correction capabilities.

[0063] Furthermore, the present invention provides an embodiment for adapting to special question types:

[0064] When grading the new countermeasures analysis questions used in the civil service examination, the system automatically adjusts the scoring weights, taking "problem diagnosis accuracy" (40%), "countermeasure innovation" (30%), and "implementation feasibility" (30%) as core dimensions, and conducts a multi-angle assessment of the "community elderly care service system" construction plan proposed by the candidates. In response to the defect of "lack of financial sustainability analysis", it generates precise suggestions such as "suggesting to increase the analysis of financing models for government procurement of services and cooperation with social capital", thus demonstrating excellent adaptability to question types.

Claims

1. A method for automatic grading of essay examinations based on a large model, characterized by: The following steps are included: S1, using supervised fine-tuning methods to train a general large language model to obtain a vertical large model suitable for the application essay correction field; S2: Input student answers, question materials, and grading standards, and use the vertical domain model to split scoring points and keyword nodes; S3, calculate the preliminary score based on the matching degree of the keyword node; S4, generate key sub-node comments and improvement suggestions; S5, performing deduplication processing on the scoring points; S6, summarizes the scores and comments, and outputs the final total score and personalized feedback.

2. The method for automatically correcting essay writing based on a large model according to claim 1 is characterized by: The training data for supervised fine-tuning includes historical question materials, student answers, grading standards and corresponding scoring results. The model is optimized through labeled data to improve its professionalism in essay grading tasks.

3. The method for automatically correcting essay writing based on a large model according to claim 1 is characterized by: The step of splitting the scoring points and keyword nodes includes splitting the student's answers into multiple scoring points according to the grading specifications, each scoring point corresponds to a scoring item, and extracting the keyword nodes of each scoring point, judging whether the keyword nodes match the requirements of the grading specifications through the vertical field big model, and calculating the preliminary score based on the degree of matching.

4. The method for automatically correcting essay writing based on a large model according to claim 1 is characterized by: The step of generating key sub-node comments and improvement suggestions includes analyzing the core information of each scoring point, evaluating its completeness and accuracy, and generating targeted comments and actionable improvement suggestions based on the correction standards.

5. The method for automatically correcting essay writing based on a large model according to claim 1 is characterized by: The deduplication process includes identifying and merging synonymous expressions of the same scoring point so that each key point is scored only once.

6. The method for automatically correcting essay essays based on a large model according to claim 1, characterized in that: The vertical field large model is built based on the LLAMA or GPT series models, and is fine-tuned to adapt to the specific needs of application essay grading, including optimizing the model's semantic understanding of government documents and policy terms.

7. The method for automatically correcting essay essays based on a large model according to claim 1, characterized in that: The method also includes a step of verifying the score calculation logic, verifying the correctness of each child node's score through a post-processing module, and checking whether there are omissions or logical errors.

8. The method for automatically correcting essay essays based on a large model according to claim 1, characterized in that: The personalized feedback includes the total score, overall comments and specific improvement suggestions for each scoring point. The overall comments summarize the advantages and disadvantages of the answer, while the sub-node suggestions provide detailed guidance, forming a multi-level feedback system.

9. The method for automatically correcting essays based on a large model according to claim 1, characterized in that: The method supports rapid grading of large-scale examinations, with the processing time for a single answer being in seconds.

10. The method for automatically correcting essays based on a large model according to claim 1, characterized in that: The method can be extended to other subjective question grading scenarios. By adjusting the training data and grading standards to adapt to different question types, its core process is universal.

Images