A method for automatic detection of code errors based on control flow graph

Abstract

The present invention relates to a method for automatically detecting code errors based on a control flow graph. The present invention generates a control flow graph according to submitted code; compares the control flow graph of the submitted code with the existing code in a database to find the most similar code in structure; determines whether the node may be wrong by calculating the code similarity of each node; and further locates the error position of the code by finding the correct code in the database that is similar to the source code logic. The present invention makes up for the shortcomings of the existing online evaluation system by giving full play to the characteristics of the control flow graph. At present, most online evaluation systems only have a code detection plagiarism system, but no automatic code error detection system. This system can help users quickly locate the error position of their own programs, thereby speeding up programming efficiency. In teaching activities, it reduces the time for teachers to check codes for students, thereby greatly reducing the burden on teachers and spending more time on teaching tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic code error detection, and relates to an automatic code error detection method based on a control flow graph. Background Art

[0002] Foreign scholars have been conducting research on text duplication detection since the 1970s, and many scholars have begun to focus on program code clone detection technology. Currently, there are mainly the following types of code cloning technology:

[0003] 1. Existing tree-based code duplication detection technology: T. Kamiya, S. Kusumoto (2002) and others found that the token-based duplication detection method is fast and requires fewer resources. However, it cannot detect all types of code clones. The tree-based duplication detection method can detect all types of code clones, but it is slow and requires a lot of computing resources. Therefore, a high-speed, high-precision code duplication detection method combining tree-based and token-based methods is proposed to improve the efficiency and accuracy of detecting code clones. Zeng Jie, Ben Kerong (2020) and others proposed a code duplication detection method based on program vector trees. Through experimental analysis, it was found that for code clones with low surface similarity of Moderately Type-3 and Type-4 types, the use of this method for detection has obvious advantages. However, this method is currently only for Java programs, and the evaluation data set BigCloneBench only contains Java programs. Therefore, migration to other programming languages ​​requires further proposal of targeted binary tree generation rules.

[0004] 2. Graph-based code duplication detection technology: Komondoor R, Horwitz S (2001) introduced the use of program dependency graph PDG (Program Dependence Graph) and program slicing to find isomorphic PDG subgraphs representing clones. This method can find discontinuous code clones, clones that reorder matching statements, and clones that are intertwined with each other. Wang Min (2018) optimized the traditional PDG-based code duplication detection technology and implemented a new PDG-based code duplication detection method based on machine learning. This method has a significant improvement in detection time, but when determining the isomorphism of the PDG subgraph, the PDG nodes are first converted into a type for matching, which causes the semantic information of some statements to be lost, and false positive results may appear in the clone determination.

[0005] 3. Code duplication detection technology based on metrics: Kontogiannis KA, DeMori R (1996) et al. proposed a dynamic programming algorithm based on source code metrics to find the best alignment between two code snippets, as well as a statistical matching algorithm between abstract code descriptions represented in an abstract language and actual source code [5]. Perumal A, Kanmani S (2010) et al. proposed a new technology for finding similar code blocks and quantifying their similarity. Their technology can be used to find clone clusters and code block sets within the similarity provided by the user. It uses type 1 and type 2 metrics to detect similar clones.

[0006] In summary, in recent years, code duplication detection methods have attracted widespread attention and attention from domestic and foreign scholars, and have achieved certain research results. However, most code duplication detection technologies are still at the research level and have not been developed into practical applications. For student assignments and code submissions for competitions, it is not only necessary to compare the submitted code with the correct code based on code duplication detection technology, but also to determine the specific error location of the code. This patent studies the submitted code for student assignments and competitions, and designs a method for automatic detection of code errors based on control flow graphs. Summary of the invention

[0007] The purpose of the present invention is to provide a method for automatically detecting code errors based on a control flow graph, which can automatically detect code errors submitted by users to an online evaluation system and facilitate users to locate error locations.

[0008] The present invention specifically comprises the following steps:

[0009] Step 1: Generate a control flow graph:

[0010] Assume that there are n codes to be tested in the database, numbered 1, 2, 3, ..., n. When compiling the code, the code numbered i generates the corresponding control flow graph G[i]. In G[i], m+1 nodes will be generated, and the jth node is N[i][j], 0≤j≤m, and its corresponding source code is C[i][j];

[0011] The compiler removes nodes that have no impact on the code logic and merges multiple nodes with single input and single output.

[0012] Step 2: Subgraph matching:

[0013] The control flow graph G[i] of the code to be tested is compared with the control flow graph G[k] generated by the correct code C[k] for subgraph isomorphism and node matching. If the control flow graphs of the two programs differ by at most one node and the two graphs are isomorphic after simple edge and point addition operations, then the two control flow graphs are considered to be matchable. Subgraph isomorphism means that given two graphs G=(V,E,u,v) and G'=(V',E',u',v'), S is a subgraph of G'. If there exists a function f:V→V ', and f is isomorphic from G to S, then f is called a subgraph isomorphism from G to G'. After the subgraph isomorphism algorithm, a set S of structural similarity is obtained. The maximum value S[k] in S is taken, indicating that code i is the most similar to code k, that is, K = argmax(S[k]);

[0014] Step 3: Branch revision:

[0015] After obtaining the relevant mapping of graph isomorphism, the subsequent operations are directly carried out. Since the specific code content of the node is not taken into consideration during the matching process, the branch correspondence of a node may be reversed, which is prone to misjudgment;

[0016] Therefore, branch correction is performed. For each branch node of the source control flow graph, the corresponding similarity detection is performed between it and the branch node of the target control flow graph using character-based text similarity calculation, and the two nodes with the highest similarity are defined as a matching relationship, so as to make the order of branches correct as much as possible and reduce the false alarm rate of error detection.

[0017] Assume that there are m+1 branch nodes in the control flow graph generated by the correct code C[k], numbered 0, 1, 2, 3, …, m. For each branch node N[k][j] (0≤j≤m) in C[k], perform LCS (Longest Common Subsequence) distance calculation between it and all branch nodes in the branch node set N[i] in the control flow graph generated by C[i]. If the similarity between N[k][j] and the p node in N[i] is greater than the similarity with other nodes in N[i], the branch node N[k][j] in k is considered to match the branch node N[i][p] in i.

[0018] Step 4: Calculate similarity based on nodes:

[0019] Edit distance is used for text similarity comparison, and LCS (Longest Common Subsequence) distance is used to define the edit distance. The length of the LCS distance source string Y is L1, the length of the target string M is L2, and the LCS distance between Y and M is EL. Then the text similarity between Y and M can be defined as: similarity = 1-EL / (L1+L2).

[0020] After obtaining the matching relationship between the two codes, we need to determine the similarity of the codes in the two nodes. The submitted code C[i] and the correct code C[k], that is, all the nodes of the source code of the two codes, are compared based on the node matching relationship based on text similarity. If the value of T[j] in the obtained text similarity set T is less than the threshold (set manually according to the requirements), it is considered that there is an error in the code C[i][j].

[0021] The present invention makes up for the shortcomings of some existing online evaluation systems by giving full play to the characteristics of control flow graphs. At present, most online evaluation systems only have code detection plagiarism systems, but no automatic code error detection system. This system can help users quickly locate the error position of their own programs, thereby speeding up programming efficiency. In teaching activities, it reduces the time teachers spend checking codes for students, thereby greatly reducing the burden on teachers and allowing them to spend more time on teaching tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the overall process flow chart of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0024] like Figure 1 As shown, a method for automatically detecting code errors based on a control flow graph is specifically as follows: a control flow graph is generated according to the submitted code; the control flow graph of the submitted code is compared with the existing code in the database to find the most similar code in structure; the code similarity of each node is calculated to determine whether the node may be erroneous; and the correct code in the database that is similar to the source code logic is found to further locate the error position of the code.

[0025] The data selected in this example are the C language codes of 480 questions submitted by 298 students. After deduplication and removal of invalid codes such as compilation failure, the data set includes 85,757 codes in total. The correct result is referred to as AC, and the wrong result is referred to as WA.

[0026] Step 1: For the submission record numbered 299383, the control flow graph code it generates is as follows:

[0027] ;; Function main (main, funcdef_no=12, decl_uid=2166, cgraph_uid=12,symbol_order=12)

[0028] Removing basic block 7

[0029] Merging blocks 5 and 6

[0030] Merging blocks 5 and 8

[0031] ;; 2 loops found ;;

[0033] ;; Loop 0

[0034] ;; header 0, latch 1

[0035] ;; depth 0, outer -1

[0036] ;; nodes: 0 1 2 3 4 5 ;;

[0038] ;; Loop 1

[0039] ;; header 4, latch 3

[0040] ;; depth 1, outer 0

[0041] ;; nodes: 4 3

[0042] ;; 2 succs { 4}

[0043] ;; 3 succs { 4}

[0044] ;; 4 succs { 3 5}

[0045] ;; 5 succs { 1}

[0046] main ()

[0047] {

[0048] double c;

[0049] double b;

[0050] double a;

[0051] int i;

[0052] int t;

[0053] int D.2180;

[0054] int t.2;

[0055] double b.1;

[0056] double a.0;

[0057] <bb 2>:

[0058] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:9:5] scanf ([ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:9:11] "%d", [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:9:5] &t);

[0059] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:11:13] i = 0;

[0060] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:11:5] goto <bb 4>;

[0061] <bb 3>:

[0062] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:17:9] scanf ([ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:17:15] "%lf %lf", [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:17:9] &a, [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:17:9] &b);

[0063] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:19:21] a.0 = a;

[0064] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:19:21] b.1 = b;

[0065] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:19:3] c = a.0 / b.1;

[0066] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:19:3] printf ([ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:19:10] "%lf\n", c);

[0067] a = {CLOBBER};

[0068] b = {CLOBBER};

[0069] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:11:22] i = i + 1;

[0070] <bb 4>:

[0071] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:11:18] t.2 = t;

[0072] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:11:5] if (i < t.2)

[0073] goto <bb 3>;

[0074] else

[0075] goto <bb 5>;

[0076] <bb 5>:

[0077] [ / mnt / c / Users / TautCony / PycharmProjects / CLC / FULL_TEST_CASE / src2017 / 1934 / WA / 299383.c:21:9] D.2180 = 0;

[0078] t = {CLOBBER};

[0079] return D.2180;

[0080] }

[0081] ;; Function printf ( <unset-asm-name>, funcdef_no=7, decl_uid=689,cgraph_uid=7, symbol_order=7)

[0082] Merging blocks 2 and 3

[0083] ;; 1 loops found ;;

[0085] ;; Loop 0

[0086] ;; header 0, latch 1

[0087] ;; depth 0, outer -1

[0088] ;; nodes: 0 1 2

[0089] ;; 2 succs { 1}

[0090] printf (const char * restrict __fmt)

[0091] {

[0092] int D.2183;

[0093] <bb 2>:

[0094] [ / usr / include / x86_64-linux-gnu / bits / stdio2.h:104:10] D.2183 = __printf_chk (1, __fmt, __builtin_va_arg_pack ());

[0095] [ / usr / include / x86_64-linux-gnu / bits / stdio2.h:104:10] return D.2183;

[0096] }

[0097] The lines starting with ";;" in GCC (GNU Compiler Collection) describe the structure of the control flow graph, and GCC has removed some unnecessary points through relevant optimization operations. For example, in the submission record numbered 299383, the useless nodes 1 and 7 have been removed, and nodes 6 and 5 have been merged (Control Flow Graph (CFG) is a method of describing programs. It is an abstract data structure used in compilers. The program will remove nodes that have no effect on the code logic and merge multiple nodes with single input and single output).

[0098] Step 2: Perform subgraph matching between the structure of the control flow graph and the control flow graph structure of the correct code under the changed question in the database, and select the most similar code. In this embodiment, the code numbered 52037 is matched;

[0099] Step 3: Compare the codes in each corresponding node. If the similarity of a node is lower than a certain threshold, here 95%, the corresponding information is output. In this embodiment, the information is as follows:

[0100] rc_wa: for(int i=0;i <t;i++){

[0101] scanf("%lf %lf",&a,&b);

[0102] printf("%lf\n",c=a / b);}

[0103] at [(19, 3), (17, 9), (11, 22), (19, 21)]

[0104] src_ac: for (i=0;i <T;i++)

[0105] scanf("%d%d",&n,&m);

[0106] printf("%.2f\n",(double)n / m);

[0107] at [(11, 16), (15, 3), (17, 6), (17, 22)]

[0108] Step 4: After all the codes to be tested are compared with the correct codes, the corresponding json files are generated, providing an interface for easy parsing.

[0109] Step 5: In order to determine whether this method is indeed effective, the data set used in the test in this embodiment includes 85,757 codes, which are manually checked. In view of the large number of codes, the codes submitted under all the questions in the data sample are matched first. For each question, the matching is stopped when 20 codes are successfully matched. Taking into account the matching time and matching rate, 200 codes of 10 questions are taken for analysis. After manual inspection, it was found that 165 codes correctly pointed out the location of the error after detection by this method, with a ratio of 83%. It can be seen that this method is practical in most cases. In most cases, the error prompts are caused by the expansion of the output range. The reason for this is that we compare and calculate each node of the control flow graph, so the error prompt will locate the range to a certain node, not a certain line of code. Therefore, although the prompt range is expanded, the prompt position still contains the actual code error location.

Claims

1. A method for automatically detecting code errors based on a control flow graph, characterized in that: The specific steps include: Step 1: Generate a control flow graph: Assume that there are n codes to be tested in the database, numbered 1, 2, 3, ..., n; when compiling the code, the code numbered i generates the corresponding control flow graph G[i], and the jth node of m+1 nodes generated in G[i] is N[i][j], 0≤j≤m, and its corresponding source code is C[i][j]; Step 2: Subgraph matching: The control flow graph G[i] of the code to be checked is matched with the control flow graph G[k] generated by the correct code C[k] for subgraph isomorphism and node matching. If the control flow graphs of the two programs differ by at most one node and the two graphs are isomorphic after simple edge and point addition operations, then the two control flow graphs are considered to be matchable. Subgraph isomorphism means that given two graphs G=(V,E,u,v) and G'=(V',E',u',v'), S is a subgraph of G'. If there exists a function f:V→V ', and f is isomorphic from G to S, then f is called a subgraph isomorphism from G to G'. After the subgraph isomorphism algorithm, a set Z of structural similarity is obtained. The maximum value Z[k] in Z is taken, indicating that code i is most similar to code k, that is, K = argmax(Z[k]). Step 3: Branch revision: Since the specific code content of the node is not taken into consideration during the matching process, the branch correspondence of a node may be reversed, which may easily lead to misjudgment; Therefore, branch correction is performed. For each branch node of the source control flow graph, the similarity between it and the branch node of the target control flow graph is detected using character-based text similarity calculation, and the two nodes with the highest similarity are defined as a matching relationship, so as to make the order of branches correct as much as possible and reduce the false alarm rate of error detection; Assume that there are m+1 branch nodes in the control flow graph generated by the correct code C[k], numbered 0, 1, 2, 3, …, m; for each branch node N[k][j] in the correct code C[k], 0≤j≤m, perform LCS distance calculation between it and all branch nodes in the branch node set N[i] in the control flow graph generated by the code to be checked C[i]; if the similarity between N[k][j] and the p node in N[i] is greater than the similarity with other nodes in N[i], it is considered that the branch node N[k][j] in the control flow graph generated by the correct code C[k] matches the branch node N[i][p] in the control flow graph generated by the code to be checked C[i]; Step 4: Calculate similarity based on nodes: Edit distance is used to compare text similarity, and LCS distance is used to define edit distance; the length of the LCS source string Y is L1, the length of the target string M is L2, and the LCS distance between Y and M is EL, then the text similarity between Y and M can be defined as: similarity = 1-EL / (L1+L2); After obtaining the matching relationship between the two codes, we need to determine the similarity of the codes in the nodes of the two codes. We compare the submitted code C[i] with the correct code C[k], that is, all the nodes of the source code of the two codes, based on the node matching relationship. We obtain the text similarity set T. If the value of T[j] is less than the threshold, it is considered that there is an error in the code C[i][j].

2. The method for automatically detecting code errors based on a control flow graph according to claim 1, characterized in that: In step 1, the compiler removes nodes that have no impact on the code logic and merges multiple nodes with single input and single output.

Images