A method and system for homework correction based on efficient human-AI collaboration

Abstract

This invention discloses a method and system for grading homework based on efficient human-AI collaboration, which lies at the intersection of artificial intelligence and educational technology. The method involves obtaining clustering features of multiple homework assignments, clustering the assignments based on the clustering features to generate #imgabs0# clusters, with each cluster #imgabs1# containing multiple assignments with similar features. For each cluster #imgabs2#, a representative assignment is selected and parameter matching is performed. For all pairs of clusters (#imgabs3#, #imgabs4#), a determination is made as to whether to merge the clusters. Finally, based on the adjusted cluster structure and the AI homework grading parameters corresponding to each cluster, batch grading is performed on the assignments within all clusters. This method and system, based on efficient human-AI collaboration, addresses the issues of poor grading consistency and low collaboration efficiency through cluster feature analysis, adaptive parameter optimization, and cluster structure adjustment, significantly reducing labor costs and improving accuracy.

Description

Technical Field

[0001] The present invention relates to the field of AI (artificial intelligence) technology, and in particular to a homework grading method and system based on efficient human-AI collaboration. Background Art

[0002] The traditional way of grading homework has the following problems:

[0003] 1. Manual grading is inefficient and inconsistent: Teachers need to check each assignment individually, which is time-consuming and labor-intensive, making it difficult to handle large-scale assignments.

[0004] 2. Low AI grading accuracy: Existing AI tools rely on fixed rules (such as regular expressions) and are unable to handle complex logic or diverse problem-solving approaches, resulting in low grading accuracy.

[0005] 3. Low efficiency of human-AI collaboration: AI and teachers lack a dynamic interaction mechanism, manual corrections cannot optimize the AI model in real time, and the clustering structure is fixed, making it difficult to adapt to changes in homework characteristics.

[0006] Existing technologies have yet to offer effective solutions to these problems. For example, patent application number 202411486847.5, titled "AI-Based Homework Grading Method and System," optimizes grading efficiency through clustering and AI parameter encapsulation, but does not address closed-loop optimization involving dynamic clustering adjustments and human feedback. Therefore, a homework grading method that supports clustering adaptive optimization and real-time human-AI collaboration is urgently needed. Summary of the Invention

[0007] The purpose of the present invention is to provide a method and system for grading homework based on efficient human-AI collaboration to solve the problems existing in the background technology.

[0008] To achieve the above objectives, the present invention provides a method and system for grading homework based on efficient human-AI collaboration, comprising the following steps:

[0009] S1. Obtain clustering features of multiple jobs, perform clustering processing on the jobs according to the clustering features, and generate clusters, each cluster ( ) contains multiple jobs with similar characteristics;

[0010] S2. For each cluster , select representative jobs and match parameters, and perform the following sub-steps:

[0011] S21, from clustering Select two representative assignments 、 ,in Compare Closer to clustering The cluster center of

[0012] S22, based on clustering Clustering features are used to match AI homework correction parameters corresponding to the most similar clusters from historical correction records;

[0013] S23, call the AI model according to the matching AI homework correction parameters, and perform the AI model on the representative homework. 、 Conduct preliminary marking;

[0014] S24. Obtain human teacher 、 Manual correction results 、 , and compare them with the AI correction results. By comparing the differences between the AI correction results and the manual correction results, the AI homework correction parameters corresponding to the cluster are dynamically adjusted;

[0015] S25, according to 、 Differences and clustering with AI correction results Number of jobs , determine whether to cluster Decomposed into two sub-clusters;

[0016] S3, for all cluster pairs ( , )( ; ), determine whether to perform cluster merging, and execute the following sub-steps:

[0017] S31. Calculate clusters and The minimum feature distance between

[0018] S32, if the minimum feature distance is lower than the preset merging threshold, and the clustering Number of jobs and clustering Number of jobs The sum of is less than the preset capacity threshold, then the cluster will be and Merge into a new cluster;

[0019] S4. Based on the adjusted cluster structure and the AI job correction parameters corresponding to each cluster, the AI model is re-called to batch correct the jobs in all clusters.

[0020] Preferably, the clustering features in step S1 include at least one of the following: a one-hot encoded feature vector of the homework type identifier, a homework complexity index, a normalized value of the homework length index, the average score of the student's historical homework, a statistical feature vector of the evaluation level or error type distribution, a paper neatness score extracted based on a convolutional neural network, and a root mean square error index of the manual and AI correction results in historical corrections.

[0021] Preferably, the specific method of matching the most similar clusters in step S22 is:

[0022] Cluster The clustering feature vector and the feature library of historical clustering are L2 normalized;

[0023] The cosine similarity is calculated using an approximate nearest neighbor search algorithm, and the AI homework grading parameters corresponding to historical clusters with a similarity greater than 0.85 are selected;

[0024] If there is no historical clustering that meets the conditions, then based on clustering The eigenvector generates the initialization parameters of the Gaussian mixture model.

[0025] Preferably, the dynamic adjustment of AI homework correction parameters in step S24 includes at least one of the following operations:

[0026] according to 、 The difference between the scores and the AI-corrected results is corrected by using the weighted least squares method to adjust the proportion of sub-item weights in the scoring weight (for example, in programming assignments, the proportion of grammatical correctness, code efficiency, style standards, etc.);

[0027] Based on the manually labeled error type priorities, the Bayesian update rule is used to adjust the logical priority order in the error feedback rule;

[0028] Based on the subjective correction results of the work presented by humans, adjust the tolerance thresholds of formatting, indentation, and specifications (for example, in programming work, adjust the tolerance thresholds of indentation and naming specifications accordingly).

[0029] Preferably, dynamic adjustment is achieved through a deep deterministic policy gradient algorithm, specifically including:

[0030] The difference vector between the manual correction result and the AI correction result As the state space, the parameter adjustment action space is defined as the continuous weight coefficient adjustment amount, and the reward function is as follows:

[0031] ;

[0032] Update the policy network parameters through the experience replay mechanism until .

[0033] Preferably, the conditions for cluster decomposition in step S25 are:

[0034] ;

[0035] in, Clustering The standard deviation of historical ratings;

[0036] Clustering The number of jobs is:

[0037] ;

[0038] in is the preset cluster size threshold.

[0039] Preferably, the conditions for cluster merging in step S32 are:

[0040] ;

[0041] ;

[0042] in, Clustering and clustering Hausdorff distance; is the feature space merging threshold; The upper limit of cluster capacity.

[0043] A homework grading system based on efficient human-AI collaboration, including:

[0044] A multimodal feature extraction module that integrates an OCR engine (to extract the neatness of handwritten papers), a code abstract syntax tree parser (to analyze grammatical structure), and a format scoring neural network;

[0045] Dynamic clustering module, including initial clustering unit based on DBSCAN algorithm, splitting unit based on spectral clustering decomposition and merging unit based on hierarchical clustering;

[0046] Parameter matching engine with built-in feature vector retrieval database (supports ANN search) and parameter generation adversarial network (generates new parameters);

[0047] Human-machine calibration interface, equipped with a double-blind annotation comparison unit and a parameter adjustment decision tree;

[0048] Adaptive optimization module, which implements a deep reinforcement learning framework based on policy gradient;

[0049] Batch correction engine that supports parallel inference pipelines accelerated by multiple GPUs.

[0050] Preferably, the human-machine calibration interface further includes:

[0051] The difference visualization unit is used to display the differences between manual and AI correction results in a graphical interface and provide editable options for parameter adjustment suggestions.

[0052] Therefore, the present invention adopts the above-mentioned homework grading method and system based on efficient human-AI collaboration, which has the following beneficial effects:

[0053] (1) Through cluster dynamic optimization (decomposition and merging), we ensure that similar assignments use the same grading strategy, reducing the standard deviation of grading results by 40% and improving grading consistency;

[0054] (2) Teachers only need to review representative assignments (5%-10% of the total), and AI completes more than 90% of standardized tasks, reducing labor costs and improving work efficiency;

[0055] (3) With enhanced adaptive capabilities and reinforcement learning driving parameter optimization, the accuracy of AI models has increased by 15%-20% during iterations.

[0056] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a schematic diagram of the overall process of a homework grading method based on efficient human-AI collaboration in the present invention;

[0058] Figure 2 This is a flow chart of clustering parameter matching and adjustment for a homework grading method based on efficient human-AI collaboration according to the present invention;

[0059] Figure 3 This is a clustering and merging flow chart of a homework grading method based on efficient human-AI collaboration in the present invention;

[0060] Figure 4 This is a schematic diagram of an assignment grading system based on efficient human-AI collaboration according to the present invention. DETAILED DESCRIPTION

[0061] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0062] See also Figure 1-Figure 3 , a homework grading method and system based on efficient human-AI collaboration, comprising the following steps:

[0063] S1. Obtain clustering features of multiple jobs, perform clustering processing on the jobs according to the clustering features, and generate clusters, each cluster ( ) contains multiple jobs with similar characteristics.

[0064] Clustering features include at least one of the following: a one-hot encoded feature vector of the homework type identifier, a homework complexity index, a normalized value of the homework length index, the average score of the student's historical homework, a statistical feature vector of the evaluation level or error type distribution, a paper neatness score extracted based on a convolutional neural network, and a root mean square error index of the manual and AI correction results in historical corrections.

[0065] S2. For each cluster , select representative jobs and match parameters, and perform the following sub-steps:

[0066] S21, from clustering Select two representative assignments 、 ,in Compare Closer to clustering The cluster center of .

[0067] S22, based on clustering The clustering features are used to match the AI homework correction parameters corresponding to the most similar clusters from the historical correction records.

[0068] The specific method of matching the most similar clusters is:

[0069] Cluster The clustering feature vector and the feature library of historical clustering are L2 normalized;

[0070] The cosine similarity is calculated using an approximate nearest neighbor search algorithm, and the AI homework grading parameters corresponding to historical clusters with a similarity greater than 0.85 are selected;

[0071] If there is no historical clustering that meets the conditions, then based on clustering The eigenvector generates the initialization parameters of the Gaussian mixture model.

[0072] S23, call the AI model according to the matching AI homework correction parameters, and perform the AI model on the representative homework. 、 Conduct preliminary revisions.

[0073] S24. Obtain human teacher 、 Manual correction results 、 , and compare them with the AI correction results. By comparing the differences between the AI correction results and the manual correction results, the AI homework correction parameters corresponding to the cluster are dynamically adjusted.

[0074] Dynamically adjusting AI homework grading parameters includes at least one of the following operations:

[0075] according to 、 The difference between the scores and the AI-corrected results is corrected by using the weighted least squares method to adjust the proportion of sub-item weights in the scoring weight (for example, in programming assignments, the proportion of grammatical correctness, code efficiency, style standards, etc.);

[0076] Based on the manually labeled error type priorities, the Bayesian update rule is used to adjust the logical priority order in the error feedback rule;

[0077] Based on the subjective correction results of the work presented by humans, adjust the tolerance thresholds of formatting, indentation, and specifications (for example, in programming work, adjust the tolerance thresholds of indentation and naming specifications accordingly).

[0078] Dynamic adjustment is achieved through a deep deterministic policy gradient algorithm, which includes:

[0079] The difference vector between the manual correction result and the AI correction result As the state space, the parameter adjustment action space is defined as the continuous weight coefficient adjustment amount, and the reward function is as follows:

[0080] ;

[0081] Update the policy network parameters through the experience replay mechanism until .

[0082] S25, according to 、 Differences and clustering with AI correction results The number of jobs ( ), determine whether to cluster Decomposed into two subclusters.

[0083] The conditions for cluster decomposition are:

[0084] ;

[0085] in, Clustering The standard deviation of historical ratings;

[0086] Clustering The number of jobs is:

[0087] ;

[0088] in is the preset cluster size threshold.

[0089] S3, for all cluster pairs ( , )( ; ), determine whether to perform cluster merging, and execute the following sub-steps:

[0090] S31. Calculate clusters and The minimum feature distance between .

[0091] S32, if the minimum feature distance is lower than the preset merging threshold, and the clustering Number of jobs and clustering Number of jobs The sum of is less than the preset capacity threshold, then the cluster will be and Merge into a new cluster.

[0092] The conditions for cluster merging are:

[0093] ;

[0094] ;

[0095] in, Clustering and clustering Hausdorff distance; is the feature space merging threshold; The upper limit of cluster capacity.

[0096] S4. Based on the adjusted cluster structure and the AI job correction parameters corresponding to each cluster, the AI model is re-called to batch correct the jobs in all clusters.

[0097] like Figure 4 , also provides a homework grading system based on efficient human-AI collaboration, including:

[0098] The multimodal feature extraction module integrates an OCR engine (to extract the neatness of handwritten assignments), a code abstract syntax tree parser (to analyze grammatical structure), and a format scoring neural network.

[0099] The dynamic clustering module includes an initial clustering unit based on the DBSCAN algorithm, a splitting unit based on spectral clustering decomposition, and a merging unit based on hierarchical clustering.

[0100] Parameter matching engine with built-in feature vector retrieval database (supports ANN search) and parameter generation adversarial network (generates new parameters).

[0101] The human-machine calibration interface is equipped with a double-blind annotation comparison unit and a parameter adjustment decision tree; it also includes a difference visualization unit, which is used to display the differences between manual and AI correction results in a graphical interface, and provides editable options for parameter adjustment suggestions.

[0102] Adaptive optimization module, implementing a deep reinforcement learning framework based on policy gradient.

[0103] Batch correction engine that supports parallel inference pipelines accelerated by multiple GPUs.

[0104] Example 1: Programming homework correction:

[0105] 1. Feature extraction:

[0106] The extracted clustering features include code AST depth (complexity), PEP8 standard score (paper neatness), and student historical error type distribution vector.

[0107] 2. Cluster generation:

[0108] Generate using DBSCAN algorithm clusters, each containing 60-120 Python jobs.

[0109] 3. Parameter matching:

[0110] Clustering The clustering feature vector is matched with the historical database, and the historical parameters with a similarity of 0.92 are selected; clustering No matching, generate Gaussian mixture model initialization parameters.

[0111] 4. Optimization of human-machine collaboration:

[0112] Teacher Correction The system updates the “code efficiency” weight (reward function R=1 / (1+|ΔS|)) through the DDPG algorithm.

[0113] 5. Clustering adjustment:

[0114] Clustering middle point( point, =100), trigger decomposition; clustering and of ( ), total number of jobs = 170 ( ), triggering a merge.

[0115] Example 2: Correction of Mathematics Handwritten Assignments:

[0116] 1. Feature extraction:

[0117] Clustering features include the number of problem-solving steps, formula complexity (LaTeX parsing), and paper neatness (CNN scoring).

[0118] 2. Dynamic Merge:

[0119] Clustering and of ( ), total number of jobs = 180 ( ), triggering a merge.

[0120] Algorithm details:

[0121] - Dynamic parameter adjustment:

[0122] Difference Vector , output parameter adjustment through DDPG strategy network :

[0123] [ ];

[0124] Where s is the current state ( ), is the policy network parameter, and σ is the exploration noise.

[0125] Therefore, this invention utilizes the aforementioned efficient human-AI collaborative homework grading method and system, which can be integrated into online education platforms (such as programming learning systems and math problem banks) and is applicable to scenarios from K12 to higher education. By reducing manual grading time by over 75%, it significantly lowers operating costs for educational institutions while improving the timeliness and accuracy of student feedback.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A homework grading method based on efficient human-AI collaboration, characterized by: The following steps are involved: S1. Obtain clustering features of multiple jobs, perform clustering processing on the jobs according to the clustering features, and generate clusters, each cluster , , contains multiple jobs with similar characteristics; S2. For each cluster , select representative jobs and perform parameter matching; The content is as follows: S21, from clustering Select two representative assignments 、 ,in Compare Closer to clustering The cluster center of S22, based on clustering Clustering features are used to match AI homework correction parameters corresponding to the most similar clusters from historical correction records; The specific method of matching the most similar clusters is: Cluster The clustering feature vector and the feature library of historical clustering are L2 normalized; The cosine similarity is calculated using an approximate nearest neighbor search algorithm, and the AI homework grading parameters corresponding to historical clusters with a similarity greater than 0.85 are selected; If there is no historical clustering that meets the conditions, then based on clustering The eigenvector generates the initialization parameters of the Gaussian mixture model; S23, call the AI model according to the matching AI homework correction parameters, and perform the AI model on the representative homework. 、 Conduct preliminary marking; S24. Obtain human teacher 、 Manual correction results 、 , and compare them with the AI correction results. By comparing the differences between the AI correction results and the manual correction results, the AI homework correction parameters corresponding to the cluster are dynamically adjusted; Dynamic adjustment is achieved through a deep deterministic policy gradient algorithm, which includes: The difference vector between the manual correction result and the AI correction result As the state space, the parameter adjustment action space is defined as the continuous weight coefficient adjustment amount, and the reward function is as follows: ； Update the policy network parameters through the experience replay mechanism until ; S25, according to 、 Difference size and clustering with AI correction results Number of jobs , determine whether to cluster Decomposed into two sub-clusters; The conditions for cluster decomposition are: ； in, Clustering The standard deviation of historical ratings; Clustering The number of jobs is: ； in is the preset cluster size threshold; S3, for all cluster pairs ( , ), , For two different clusters among all clusters; determine whether to merge the clusters; S4. Based on the adjusted cluster structure and the AI job correction parameters corresponding to each cluster, the AI model is re-called to batch correct the jobs in all clusters.

2. A method for grading homework based on efficient human-AI collaboration according to claim 1, characterized in that: The clustering features in step S1 include at least one of the following: a one-hot encoded feature vector of the homework type identifier, a homework complexity index, a normalized value of the homework length index, the average score of the student's historical homework, a statistical feature vector of the evaluation level or error type distribution, a paper neatness score extracted based on a convolutional neural network, and a root mean square error index of the manual and AI correction results in historical correction.

3. The method for grading homework based on efficient human-AI collaboration according to claim 1, characterized in that: The S3 content is as follows: S31. Calculate clusters and The minimum feature distance between S32, if the minimum feature distance is lower than the preset merging threshold, and the clustering Number of jobs and clustering Number of jobs The sum of is less than the preset capacity threshold, then the cluster will be and Merge into a new cluster.

4. The method for grading homework based on efficient human-AI collaboration according to claim 3, characterized in that: The conditions for cluster merging in step S32 are: ； ； in, Clustering and clustering Hausdorff distance; is the feature space merging threshold; The upper limit of cluster capacity.

5. A homework grading system based on efficient human-AI collaboration, applying the homework grading method based on efficient human-AI collaboration according to any one of claims 1 to 4, characterized in that: include: Multimodal feature extraction module, integrating OCR engine, code abstract syntax tree parser and format scoring neural network; Dynamic clustering module, including initial clustering unit based on DBSCAN algorithm, splitting unit based on spectral clustering decomposition and merging unit based on hierarchical clustering; Parameter matching engine with built-in feature vector retrieval database and parameter generation adversarial network; Human-machine calibration interface, equipped with a double-blind annotation comparison unit and a parameter adjustment decision tree; Adaptive optimization module, which implements a deep reinforcement learning framework based on policy gradient; Batch correction engine that supports parallel inference pipelines accelerated by multiple GPUs.

6. The homework grading system based on efficient human-AI collaboration according to claim 5 is characterized in that: The human-machine calibration interface also includes: The difference visualization unit is used to display the differences between manual and AI correction results in a graphical interface and provide editable options for parameter adjustment suggestions.

Images