A student homework correction method based on submission history information

Abstract

The application discloses a student homework correction method based on submission history information, which comprises the following steps: firstly, processing the submission history information, extracting the latest and initial code changes of the student, extracting the submission message and the code changes of the reference program each time, segmenting the code changes extracted from the student and the reference program, and establishing code change segments; then, finding the code change segments that may be matched between the student program and the reference program, calculating the similarity after finding the code change segments, and establishing a similarity set; then, finding the best matching code segment by using the maximum weight algorithm of the bipartite graph; then, performing text similarity calculation on the submission message corresponding to the matched code segment; and finally, integrating the similarity of the code segment and the corresponding message according to the proportion.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, especially software technology, and specifically proposes a method for grading student assignments based on submission history information. Background Technology

[0002] With the rapid development of internet technology and the ever-changing software industry, computer science has become a popular major, attracting an increasing number of students. However, this also means that teachers need to spend a significant amount of time and energy grading student assignments, which undoubtedly poses a challenge. While automated evaluation of programming assignments has been researched for many years and numerous tools have been developed, most such systems are test-based. Generally, researchers focus on checking functional correctness; typically, automated evaluation is accomplished by examining the program's behavior on a set of test inputs. Test cases can be created manually or generated automatically. In proposed systems, inputs that fail the test are provided as feedback. However, generating a set of test inputs that can cover all possible errors submitted by students is impossible, and online platforms for these programming assignments cannot meet the needs of traditional school teaching.

[0003] As code repositories continue to evolve, software projects can be monitored through collaborative platforms. Services like GitHub make student activities transparent and enable continuous monitoring of project progress and student participation. In the industry and open-source software development field, various metrics have been proposed to quantify the contributions of developers using source code repositories. Summary of the Invention

[0004] This invention proposes a method for effectively grading student programs by processing reference and student programs based on submission history information and coarse-grained tracking of the student submission process. First, this invention focuses on the submission history information of student and reference programs, including code changes and submission messages. The acquired code changes are processed by classifying them according to filename and modification type (addition or deletion), and then segmenting the code changes at the single-line statement level, forming a set of segmented code change fragments. Next, the segmented set of reference program code change fragments is traversed. For each code change fragment, a matching code change fragment is searched in the segmented set of student code change fragments according to file path and modification type, and the similarity is calculated. Then, a bipartite graph maximum weight algorithm is used to find the best match for each segmented code change fragment. Finally, the similarity of the corresponding submission messages is calculated for these matching code change fragments. For student code change fragments, the submission time and submission message text content need to be identified.

[0005] A method for grading student assignments based on submission history information includes the following steps:

[0006] Step 1: Process the submission history information, extract the student's latest and initial code changes, and extract the submission messages and code changes of the reference program for each submission.

[0007] Step 2: Segment the code changes extracted from the student and reference programs to create code change segments;

[0008] Step 3: Find matching code change segments between the student program and the reference program, calculate the similarity after finding them, and build a similarity set;

[0009] Step 4: Use the maximum weight algorithm of bipartite graphs to find the best matching code snippet;

[0010] Step 5: Calculate the text similarity of the submission messages corresponding to the matched code snippets;

[0011] Step 6: Integrate the similarity between the code snippets and their corresponding messages according to the ratio.

[0012] Furthermore, in step 1, when extracting code changes, the `git log --name-only` command is used to retrieve the entire commit history of the student program and the files modified in each instance. For each file in each commit, a complete record of information is generated (commit hash, message, file path). After deduplication, the latest and initial commits for each modified file are obtained and then analyzed using `gitdiff`.<commitId1,commitId2> Retrieve the code changes between two versions, where commitId1 represents the hash value of the latest commit and commitId2 represents the hash value of the initial commit.

[0013] Furthermore, in step 2, the obtained student and reference program code changes are processed again, classified according to file name and modification type, and the code changes are divided into single-line statement levels. The complete information of each line of code change is <file name, modification type, statement text>, forming a set of segmented code change fragments.

[0014] Furthermore, in step 3, when calculating similarity, the set of code change fragments of the segmented reference program is traversed. For each code change fragment, a matching code change fragment is searched in the set of segmented student code change fragments according to the file path and modification type. After finding it, the code change fragment is processed as text. First, the corpus is segmented into words, and punctuation marks need to be processed. The processed word text is used to train the model using the word2vec module. After training, each token will be mapped to a space of the same dimension, and the cosine similarity between the two vectors is calculated.

[0015] Further, in step 4, the sets of code changes for students and reference programs are denoted as A and B, where the similarity between Ai and Bj is used as the weight of the corresponding edge between Ai and Bj in the bipartite graph. The weighted bipartite graph is considered as a bipartite graph where there is an edge from every vertex of the X set to every vertex of the Y set, and it must have a perfect matching. In order to maximize the weight, a greedy algorithm is used to select edges to form a new bipartite graph, and the maximum matching is found on the basis of this bipartite graph to determine the best matching.

[0016] Furthermore, in step 5, for the two matched code change fragments, the corresponding submission message content is found. For the student program code change fragment, it is necessary to traverse the student's submission history, start from the earliest submission for precise matching, locate when the submission was made and the submission message content, and then use word2vec to convert it into a vector to calculate the cosine similarity.

[0017] Furthermore, in step 6, the text similarity of the matched code change fragments and the similarity of the corresponding submission messages are combined proportionally to calculate the final similarity of each code change fragment between the student program and the reference program. First, the average similarity of the code change fragments is calculated based on the similarity of the matched code change fragments, denoted as s1. Then, the average similarity of the submission messages is calculated based on the content similarity of the corresponding submission messages, s2. Finally, the final similarity is calculated proportionally. In this invention , .

[0018] In response to the above problems, the present invention achieves the following effects: the present invention can track the student submission process in a coarse-grained manner, thereby effectively grading student programs, while helping teachers improve the efficiency of grading assignments and reduce the investment of human, material and financial resources. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating code changes based on commit history in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram illustrating code fragment similarity calculation and matching based on text processing in an embodiment of the present invention.

[0021] Figure 3 The image shows a student program that implements the string copying function, along with a sample version of the reference program. Detailed implementation details are provided.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0023] This invention proposes a method for grading student assignments based on submission history information, specifically including the following steps:

[0024] Step 1: Initial processing is performed on the student and reference programs. For the student program, a list of changed files is collected, and for each changed file, the latest and initial commits are obtained, capturing the code changes between the two versions. For the reference program, the code changes for each commit and the commit information are obtained. The student program is then identified as the student's submitted assignment, and the reference program is identified as the reference answer to the assignment.

[0025] Step 2 involves further processing the obtained student and reference program code changes, classifying them according to file name and modification type (adding code, deleting code), and then segmenting the code changes at the single-line statement level to form a set of segmented code change fragments.

[0026] Step 3: Traverse the set of code change segments of the segmented reference program. For each code change segment, search for possible matching code change segments in the set of code change segments of the segmented student program according to the file path and modification type. After finding them, process the code change segments as text, convert them into vectors using word2vec, and then calculate the cosine similarity to form the code change segment similarity.

[0027] Step 4, denote the sets of code changes for the student and the reference program as A and B, where A i With B j The similarity between them is used as A in the bipartite graph. i With B j Find the best match by considering the weights of the corresponding edges.

[0028] Step 5: For the matched code change fragment pairs, find the corresponding submission message content. For student program code change fragments, this requires traversing the student's submission history to locate when the submission occurred and the submission message content, then using word2vec to convert it into a vector and calculate the cosine similarity.

[0029] Step 6: Finally, the text similarity of the matched code change fragments and the similarity of the corresponding submission messages are combined proportionally to calculate the final similarity of each code change fragment between the student program and the reference program.

[0030] Specifically, for each step:

[0031] In step 1, the student program retrieves all commit history and the files modified in each instance using "git log --name-only". Each file in each commit corresponds to a complete entry (commit hash, message, file path). After deduplication, the program retrieves the latest and initial commits for each modified file and then uses "git diff".<commitId1,commitId2> "To obtain the code changes between two versions, and for the reference program, to obtain the code changes and commit information for each commit."

[0032] In step 2, the obtained student and reference program code changes are processed again, and they are classified according to file name and modification type (adding code, deleting code). The code changes are then segmented at the single-line statement level. The complete information of each line of code change is <file name, modification type, statement text>, forming a set of segmented code change fragments.

[0033] In step 3, the set of code change fragments of the segmented reference program is traversed. For each code change fragment, a possible matching code change fragment is searched in the set of code change fragments of the segmented student program according to the file path and modification type. After finding it, the code change fragment is processed as text. The code change fragment text is used as a corpus for preprocessing. First, the corpus is segmented into words and separated by spaces. After segmentation, stop words are removed. The processed word text is used to train the model using the word2vec module. After training, each token will be mapped to the same dimension space, corresponding to an n-dimensional vector. Then, the cosine similarity between the two vectors is calculated according to formula (1).

[0034] (1)

[0035] In step 4, the sets of students and the changed code segments of the reference program are denoted as A and B, respectively. The similarity between Ai and Bj is used as the weight of the corresponding edge between Ai and Bj in the bipartite graph. The weighted bipartite graph can be regarded as a bipartite graph where there is an edge from every vertex of the X set to every vertex of the Y set (adding edges that were not originally in the bipartite graph with a weight of 0 is sufficient). That is to say, it must have a perfect matching (i.e., its matching number is min(|X|,|Y|)). Let M be a perfect matching of a weighted bipartite graph. Given a vertex label for each vertex, let the vertex label of Ai be... The one in Bj is If all edges They all , It is the edge The weights, if for any They all Therefore, it can be proven that M is an optimal matching. First, initialization is required. The maximum weight of the edge associated with Ai. If all values ​​are 0, then the Hungarian algorithm is used for matching. If the Hungarian algorithm can find a match... If the conditions are perfectly matched, then the best matching is found. However, during the search, it may be impossible to find such a perfect matching; only the current maximum matching may be found. In this case, the KM algorithm is needed to update the vertex indices, allowing the search for the best matching to continue. Assuming that the point already traversed in A is p, and the point already traversed in B is q, then the current minimum difference d is updated. Then we can obtain the solution formula for d. Then update the top label. Then, the Hungarian algorithm is used for matching, and the above process is repeated until the best match is found.

[0036] In step 5, for the two matched code change fragments, the corresponding submission message content is found. For the student's program code change fragment, it is necessary to traverse the student's submission history, starting from the earliest submission for precise matching, to locate when the submission was made and the submission message content, and then use word2vec to convert it into a vector and calculate the cosine similarity.

[0037] In step 6, the text similarity of the matched code change fragments and the similarity of the corresponding submission messages are combined proportionally to calculate the final similarity of each code change fragment between the student program and the reference program. First, the average similarity of the code change fragments is calculated based on the similarity of the matched code change fragments, denoted as s1. Then, the average similarity of the submission messages is calculated based on the content similarity of the corresponding submission messages, denoted as s2. Finally, the final similarity is calculated proportionally. In this embodiment , .

[0038] The work and contributions of this invention are as follows:

[0039] 1. Code change segmentation based on commit history. For student programs, collect a list of files whose changes have been made. For each changed file, obtain the latest and initial commits, capturing the code changes from both versions. For reference programs, obtain the code changes and commit information for each commit. The obtained code changes from both student and reference programs are then processed further, categorized by filename and modification type (addition or deletion). The code changes are then segmented at the single-line statement level, forming a set of segmented code change fragments.

[0040] 2. Calculation of code fragment similarity based on text processing. Traverse the set of segmented reference code variation fragments. For each code variation fragment, search for a matching code variation fragment in the set of segmented student code variation fragments, based on file path and modified type. Once found, process the code variation fragment as text using word2vec to convert it into a vector, and then calculate the cosine similarity to form the code variation fragment similarity.

[0041] 3. Code change segment matching based on the bipartite graph maximum weight algorithm. Let A and B be the sets of code change segments for students and for reference, respectively, where A... i With B j The similarity between them is used as A in the bipartite graph. i With B j Find the best match by considering the weights of the corresponding edges.

[0042] 4. Similarity calculation of submission messages based on text processing. For the matched code change fragment tuples, find the corresponding submission message content. For student version code change fragments, it is necessary to traverse the student's submission history to locate when the submission was made and the submission message content, and then use word2vec to convert it into a vector and calculate the cosine similarity.

[0043] Figure 3 This document presents a student program and a reference program example to implement the string copying function. First, the text of both programs is divided into line segments. The student program code segment is denoted as A(1,2,…,m), and the reference program code segment as B(1,2,…,n). These code segments are then converted into vectors using word2vec, denoted as Vs(1,2,…,m) and Vt(1,2,…,n), respectively. The similarity scores are calculated and denoted as Ss(1,2,…,m) and St(1,2,…,n). Finally, the maximum weighted matching algorithm is used to calculate the set of best-matching segments. The length of this set is denoted as... Let Sm(1,2,…,l) denote the similarity of the best matching fragment set. Then, use word2vec to calculate the cosine similarity Sc(1,2,…,l) of the submission message corresponding to the best matching fragment. Finally, calculate the average similarity of Sm and Sc and denote it as s1 and s2, respectively. The final similarity is then calculated. .

[0044] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A method for grading student assignments based on submission history information, characterized in that: The method includes the following steps: Step 1: Process the submission history information, extract the student's latest and initial code changes, and extract the submission messages and code changes of the reference program for each submission. Step 2: Segment the code changes extracted from the student and reference programs to create code change segments; Step 3: Find matching code change segments between the student program and the reference program, calculate the similarity after finding them, and build a similarity set; Step 4: Use the maximum weight algorithm of bipartite graphs to find the best matching code snippet; In step 4, the sets of students and the changed code segments of the reference program are denoted as A and B, respectively. The similarity between Ai and Bj is used as the weight of the corresponding edge between Ai and Bj in the bipartite graph. The weighted bipartite graph is considered as a bipartite graph where every vertex of the X set has an edge to every vertex of the Y set, meaning it must have a perfect matching, i.e., its matching number is min(|X|,|Y|). Let M be a perfect matching of the weighted bipartite graph, and given a vertex label for each vertex, let the vertex label in Ai be... The one in Bj is If all edges They all , It is the edge The weights, if for any They all To prove that M is an optimal matching, we first need to initialize it. The maximum weight of the edge associated with Ai. If all values ​​are 0, then the Hungarian algorithm is used for matching. If the Hungarian algorithm can find a match... If a perfect match is found, then the optimal match is found. If no such perfect match can be found during the search, then the current maximum match is found. In this case, the KM algorithm is needed to update the vertex indices, allowing the search for the optimal match to continue. Assuming the point already traversed in A is p, and the point already traversed in B is q, then the current minimum difference d is updated. Then we obtain the solution formula for d. Then update the top label. Then the Hungarian algorithm is used for matching, and the above process is repeated until the best match is found. Step 5: Calculate the text similarity of the submission messages corresponding to the matched code snippets; Step 6: Integrate the similarity between the code snippets and their corresponding messages according to the ratio.

2. The method for grading student assignments based on submission history information according to claim 1, characterized in that: In step 1, when extracting code changes, the `git log --name-only` command is used to retrieve the entire commit history of the student program and the files modified in each instance. For each file in each commit, a complete entry is created (commit hash, message, file path). After deduplication, the latest and initial commits are retrieved for each changed file, and then analyzed using `git diff`.<commitId1,commitId2> Retrieve the code changes between two versions, where commitId1 represents the hash value of the latest commit and commitId2 represents the hash value of the initial commit.

3. The method for grading student assignments based on submission history information according to claim 1, characterized in that: In step 2, the obtained student and reference program code changes are processed again. They are classified according to file name and modification type, and the code changes are divided into single-line statement levels. The complete information of each line of code change is <file name, modification type, statement text>, forming a set of segmented code change fragments.

4. The method for grading student assignments based on submission history information according to claim 1, characterized in that: In step 3, when calculating similarity, the set of code change fragments of the segmented reference program is traversed. For each code change fragment, a matching code change fragment is searched in the set of segmented student code change fragments according to the file path and modification type. After finding it, the code change fragment is processed as text. First, the corpus is segmented into words, and punctuation marks need to be processed. The processed word text is used to train the model using the word2vec module. After training, each token will be mapped to a space of the same dimension, and the cosine similarity between the two vectors is calculated.

5. A method for grading student assignments based on submission history information according to claim 1, characterized in that: In step 5, for the two matched code change fragments, the corresponding submission message content is found. For the student program code change fragment, it is necessary to traverse the student's submission history, start from the earliest submission for precise matching, locate when the submission was made and the submission message content, and then use word2vec to convert it into a vector to calculate the cosine similarity.

6. The method for grading student assignments based on submission history information according to claim 1, characterized in that: In step 6, the text similarity of the matched code change fragments and the similarity of the corresponding submission messages are combined proportionally to calculate the final similarity of each code change fragment between the student program and the reference program. First, the average similarity of the code change fragments is calculated based on the similarity of the matched code change fragments, denoted as s1. Then, the average similarity of the submission messages is calculated based on the content similarity of the corresponding submission messages, denoted as s2. Finally, the final similarity is calculated proportionally. .

Images