File code similarity analysis method based on directed graph isomorphism

Abstract

The invention discloses a file code similarity analysis method, which is based on a directed graph isomorphism theory and comprises the following steps of: A, extracting characteristics of an open source code file, namely a Call Graph (Call Graph for short), and establishing a file sample library; b, extracting a function call relation graph of the file to be analyzed; c, executing the standardization work of the graph, and preprocessing the graph according to the related definition of the tree structure; d, extracting the maximum common subgraph of the function call relation graph of the file to be analyzed and the sample library file; and E, calculating a relationship between the maximum common subgraph and the function call relationship graph to obtain a similar result, and completing file code similarity analysis. According to the method, the similarity of the file codes is analyzed by depending on the function call relation graph, the internal characteristics and higher-level logic characteristics of the file codes are considered, and higher accuracy and efficiency are achieved. The method is suitable for occasions such as source code plagiarism detection.

Description

Technical Field

[0001] This invention relates to software technology for analyzing file code similarity, and more particularly to a method for determining the similarity of program files based on the theory of directed graph isomorphism. Background Technology

[0002] Code similarity detection technology evolved from duplicate code detection and code optimization techniques. Similarity goes beyond simply copying files and making minor layout modifications; it also includes synonym substitution and restatement. The development of this type of technology has mainly gone through two stages: attribute counting-based and structure-based measurement-based.

[0003] Structure-based similarity detection methods measure the similarity between two programs according to their structure. The detection mechanism mainly includes: first, feature extraction of the program code, performing lexical analysis to generate token strings that express the meaning of the program code; and second, similarity calculation of the generated token strings to determine their similarity score.

[0004] After studying several existing similarity detection tools based on structure metrics, we found the following two main problems: 1. The time and space efficiency of code feature extraction is low, making it difficult to apply to large-scale software systems. 2. They reflect less of the program's syntactic and semantic information, resulting in low program similarity detection performance and requiring more manual analysis of the reported results.

[0005] Therefore, we propose a method for file code similarity analysis based on the theory of directed graph isomorphism. Summary of the Invention

[0006] The main objective of this invention is to propose a method for file code similarity analysis based on the theory of directed graph isomorphism, thereby improving both time and space efficiency and accuracy. To achieve this objective, the technical solution of this invention is implemented as follows:

[0007] A file code similarity analysis method, based on the theory of directed graph isomorphism, includes the following steps:

[0008] A. Extract the features of open-source code files—function call graphs (CG)—and build a file sample library;

[0009] B. Extract the function call graph from the file to be analyzed. CG is a directed graph that reflects the functional logic control flow. Nodes in the graph represent functions in the file, and edges represent the function call relationships.

[0010] C. Standardize the execution graph by preprocessing it according to the relevant definitions of tree structures. Define the function call graph as an acyclic directed graph with a single, fixed entry node and exit node;

[0011] D. Extract the largest common subgraph of the function call relationship graph between the file to be analyzed and the sample library files;

[0012] E. Calculate the relationship between the maximum common subgraph and the function call graph to obtain similarity results and complete the file code similarity analysis.

[0013] In the above technical solution, content B includes the following:

[0014] B1. A function call graph is a directed graph that reflects the logical control flow of functions. The nodes (Vexnode) in the graph represent functions in the file, and the edges (Edge) represent the function call relationships.

[0015] B2. The nodes in the diagram include two features for a function: structural feature and functional feature.

[0016] In the above technical solution, the content B2 is characterized by:

[0017] B2.1 Structural features include instruction information, measurement information, control flow, and other information.

[0018] B2.2 Functional characteristics can represent the logical relationship between the input and output of a function.

[0019] B2.3 When functions have similar structural features or similar functional features, functions f1 and f2 can be considered similar, denoted as equals(f1,f2).

[0020] In the above technical solution, the content C is characterized by:

[0021] C1. The function call graph of a file is actually a directed graph. When a function in the file has no dependency on other functions, an isolated node is created. In this case, the file is not a single-entry file, and a virtual node is artificially added as a unified entry point. The entry node is then taken as the root node of the tree.

[0022] C2. When recursive or circular calls exist between functions, cycles will be generated in the graph. For graphs containing self-loops, a new node needs to be added to the edge of the self-loop, splitting the self-loop edge into two directed edges. It is easy to prove that this treatment does not affect the isomorphism between the two graphs.

[0023] For a directed graph containing natural cycles, perform a depth-first traversal starting from the entry node, while logically maintaining a tree structure with the entry node as the root node and deleting redundant edges. The resulting graph is an acyclic directed graph.

[0024] C3. The acyclic directed graph generated by referring to the tree structure is a graph with a non-single exit node, and a virtual node needs to be added manually as a unified exit.

[0025] In the above technical solution, the content D specifically includes:

[0026] D1. Let graph H be the maximum common subgraph of graphs G1 and G2. Then it means that all common nodes and all common edges of graphs G1 and G2 constitute graph H.

[0027] In the above technical solution, the content D1, which generates the maximum common subgraph, specifically includes the following steps:

[0028] D1.1 Find the common node of graphs G1 and G2. Traverse all nodes of graphs G1 and G2 respectively. If there exist (v1→f1)∈G1, (v2→f2)∈G2 and equals(f1,f2), then v1(v2)∈H, that is, v1(v2) is a common node of graphs G1 and G2.

[0029] D1.2 Find the common edge between graphs G1 and G2. Take any two nodes vi and vj ∈ H. If there exists e(vi,vj) ∈ G1 and e(vi,vj) ∈ G2, then an edge e(vi,vj) ∈ H is generated.

[0030] D1.3, End of generating the maximum common subgraph

[0031] In the above technical solution, the content E specifically includes:

[0032] E1. Let the sample library file be F1, and its corresponding function call relationship graph be G1; let the file to be tested be F2, and its corresponding function call relationship graph be G2. When G1 and G2 have the three relationships of equivalence, implication, and intersection, it can be determined that the file codes are similar.

[0033] E2. Based on the above content D1, the greatest common subgraph of G1 and G2 is H. Analyze the isomorphism between H and G1, and between H and G2, to determine the relationship between G1 and G2.

[0034] In the above technical solution, the content E1 specifically includes the following:

[0035] E1.1 Equivalence; When graphs G1 and G2 are isomorphic and corresponding nodes in the graphs are similar (i.e., corresponding functions in the two files are similar), the two files can be considered equivalent. This can resolve simple similarities such as copying line by line and changing comments and blank lines between files.

[0036] E1.2 Implication; When graph G1 is a subgraph of graph G2, or vice versa, and corresponding nodes in the graphs are similar (i.e., corresponding functions in the two files are similar), then an implication relationship exists between the files. In this case, the two files can be considered structurally similar. It can be assumed that the code has the same logical relationship, only changing information such as variable names, data types, and declaration locations.

[0037] E1.3 Intersection; H is the largest common subgraph of graphs G1 and G2. The larger H is, the more intersections graphs G1 and G2 have, and the more similar the two graphs are. When H is greater than a certain threshold, the two graphs can be considered similar, and F1 and F2 intersect. In this case, the two files can be judged to be functionally similar. This can be analyzed by including redundant or unreachable statements in the code, adjusting the order of statements, and addressing similar cases where the code performs the same function.

[0038] In the above technical solution, content E2 specifically includes the following:

[0039] E2.1 First, determine whether the maximum common subgraph H is isomorphic to G1;

[0040] E2.2 Next, determine whether the largest common subgraph H is isomorphic to G2; if H is isomorphic to both G1 and G2, it means that G1 and G2 are equivalent, and the file code is similar; if H is isomorphic to either G1 or G2, it means that G1 and G2 have an implication relationship, and the file code is similar.

[0041] E2.3 If the maximum common subgraph H is not structurally similar to G1 and G2, then determine whether H is greater than a preset threshold. If it is greater than the preset threshold, it means that G1 and G2 have an intersection relationship, and the file code can be determined to be similar. If it is less than the preset threshold, it means that G1 and G2 do not have an intersection relationship, and the file code can be determined to be dissimilar.

[0042] E2.5, End. Attached Figure Description

[0043] Figure 1 The process of file code similarity analysis;

[0044] Figure 2 A function call diagram for a specific implementation case;

[0045] Figure 3 The process of graph preprocessing;

[0046] Figure 4 A standardized flow of function call relationship diagrams for a specific implementation case;

[0047] Figure 5 Process for determining file code similarity. Detailed Implementation

[0048] The present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0049] Figure 1 The process of file code similarity analysis is given:

[0050] A. Extract the function call graphs from open-source code files and create a file sample library;

[0051] B. Extract the function call graph of the file to be analyzed;

[0052] C. Standardization of execution diagrams;

[0053] D. Extract the maximum common subgraph;

[0054] E. Analyze the similarity of file codes.

[0055] As shown below, the F1 code for file F1 (using C language as an example) is:

[0056] static void script_client_vars_init(){

[0057] ……;script_client_vars_free();……}

[0058] static void script_client_vars_free(){

[0059] ……;script_client_vars_load();……}

[0060] static void script_client_vars_load(){

[0061] ……;script_client_vars_init();……}

[0062] static void script_client_vars_save(){

[0063] ……;script_client_vars_save();……}

[0064] As mentioned earlier, before file similarity analysis, it is necessary to extract the function call graph of the file to be analyzed, i.e., step B includes:

[0065] B1. A function call graph is a directed graph that reflects the logical control flow of functions. The nodes (Vexnode) in the graph represent functions in the file, and the edges (Edge) represent the function call relationships.

[0066] B2. The nodes in the diagram include two features for a function: structural feature and functional feature.

[0067] Following step B above, the function call relationship diagram for F1 can be obtained as follows: Figure 2 As shown:

[0068] As mentioned earlier, before performing file similarity analysis, it is necessary to standardize the extracted file function call relationship graph. Figure 3 The steps for standardizing execution diagrams are given below:

[0069] C1. The function call graph of a file is actually a directed graph. When a function in the file has no dependency on other functions, an isolated node is created. In this case, the file is not a single-entry file, and a virtual node is artificially added as a unified entry point. The entry node is then taken as the root node of the tree.

[0070] C2. When recursive or circular calls exist between functions, cycles will be generated in the graph. For graphs containing self-loops, a new node needs to be added to the edge of the self-loop, splitting the self-loop edge into two directed edges. It is easy to prove that this treatment does not affect the isomorphism between the two graphs.

[0071] For a directed graph containing natural cycles, perform a depth-first traversal starting from the entry node, while logically maintaining a tree structure with the entry node as the root node and deleting redundant edges. The resulting graph is an acyclic directed graph.

[0072] C3. The acyclic directed graph generated by referring to the tree structure is a graph with a non-single exit node, and a virtual node needs to be added manually as a unified exit.

[0073] Figure 4 The standardized operation process and result diagrams of the above specific implementation methods are given.

[0074] Figure 5 The specific steps for determining file code similarity are given:

[0075] E2.1 First, determine whether the maximum common subgraph H is isomorphic to G1;

[0076] E2.2 Next, determine whether the largest common subgraph H is isomorphic to G2; if H is isomorphic to both G1 and G2, it means that G1 and G2 are equivalent, and the file code is similar; if H is isomorphic to either G1 or G2, it means that G1 and G2 have an implication relationship, and the file code is similar.

[0077] E2.3 If the maximum common subgraph H is not structurally similar to G1 and G2, then determine whether H is greater than a preset threshold. If it is greater than the preset threshold, it means that G1 and G2 have an intersection relationship, and the file code can be determined to be similar. If it is less than the preset threshold, it means that G1 and G2 do not have an intersection relationship, and the file code can be determined to be dissimilar.

[0078] E2.5, End.

[0079] In the experiment, the best results were achieved when the preset threshold was 0.7. This completes all the operations for file code similarity analysis.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A file code similarity analysis method based on the theory of directed graph isomorphism, comprising the following steps: A. Extracting the features of open source code files - function call graphs (CG), and establishing a file sample library; B. Extracting the function call graph of the file to be analyzed. CG is a directed graph that reflects the logical control flow of functions, with nodes representing functions in the file and edges representing the call relationship between functions; C. Performing standardization of the graph, and pre-processing the graph according to the relevant definitions of tree structure. Define the function call graph as an acyclic directed graph with a single, fixed entry node and exit node; D. Extracting the largest common subgraph (LCSG) of the function call graphs of the file to be analyzed and the sample library files; E. Calculating the relationship between the largest common subgraph and the function call graph to obtain the similarity result and complete the file code similarity analysis.

2. The method of claim 1, wherein, Content B includes the following: B1. The function call graph is a directed graph that reflects the logical control flow of functions, with nodes (Vexnode) representing functions in the file and edges (Edge) representing the call relationship between functions. B2. The nodes in the graph include two characteristics of the function, namely structural features (Struct Feature) and functional features (Function Feature).

3. The method of claim 2, wherein, The content B2 is characterized by: B2.

1. The structural features include instruction information, metric information, control flow, etc. B2.

2. The functional features can represent the logical relationship between the input and output of a function. B2.

3. When the structural features of functions f1 and f2 are similar or the functional features are similar, it can be considered that the functions f1 and f2 are similar, denoted as equals(f1, f2).

4. The method of claim 1, wherein, The content C is characterized by: C1. The function call graph of a file is actually a directed graph, and when a function in the file does not have a dependency relationship with other functions, an isolated node will be generated. At this time, the file is a non-single entry file, and a virtual node is artificially added as a unified entry. The entry node is taken as the root node of the tree. C2. When there is recursive or circular calling between functions, a loop will be generated in the graph. For a graph containing a self-loop, a node is added to the edge of the self-loop, dividing the self-loop edge into two directed edges. It is easy to prove that this processing will not affect the isomorphism between the two graphs. For a directed graph with a natural loop, perform a depth-first traversal from the entry node, and logically maintain a tree structure with the entry node as the root node. Remove the redundant edges, and the resulting graph is an acyclic directed graph. C3. The acyclic directed graph generated by referring to the tree structure is a graph with a non-single exit node, and a virtual node is artificially added as a unified exit.

5. The method of claim 1, wherein, Content D specifically includes: D1. Let H be the largest common subgraph of graphs G1 and G2, then H represents all common nodes and all common edges of graphs G1 and G2.

6. The method of claim 1, wherein, Content D1, which generates the largest common subgraph, specifically includes the following steps: D1.1, solve the common nodes of graph G1 and graph G2. Traverse all nodes of graph G1 and graph G2 respectively, if (v1→f1)∈G1, (v2→f2)∈G2 and equals(f1,f2) exist, then v1(v2)∈H, that is, v1(v2) is the common node of graph G1 and graph G2. D1.2, solve the common edges of graph G1 and graph G2. Take any two nodes vi and vj∈H, if e(vi,vj)∈G1 and e(vi,vj)∈G2 exist, then an edge e(vi,vj)∈H is generated. D1.3, generate the maximum common subgraph.

7. The method of claim 1, wherein, The content E specifically includes: E1, set the sample library file as F1, and its corresponding function call relationship graph as G1; the file to be tested as F2, and its corresponding function call relationship graph as G2. When G1 and G2 have the three relationships of equivalence, implication and intersection, it can be determined that the file codes are similar; E2, obtain the maximum common subgraph H of G1 and G2 from the above content D1. Analyze the isomorphism of H and G1, H and G2 respectively to determine the relationship between G1 and G2.

8. The method of claim 7, wherein, The content E1 specifically includes the content: E1.1, equivalence; when the two graphs G1 and G2 are isomorphic, and the corresponding nodes in the graph are similar (that is, the corresponding functions in the two files are similar), it can be considered that the two files are equivalent. In this case, it can solve the simple similarity between files such as sentence by sentence copy, changing comments and empty lines; E1.2, implication; when graph G1 is a subgraph of graph G2, or vice versa, and the corresponding nodes in the graph are similar (that is, the corresponding functions in the two files are similar), it is considered that there is an implication relationship between the two files. At this time, the two files can be judged to be structurally similar. It can be considered that the codes have the same logical relationship, only the variable name, data type and declaration position information are changed; E1.3, intersection; H is the maximum common subgraph of graph G1 and graph G2. The larger H is, the more intersection G1 and G2 have, and the more similar the two graphs are. When the H graph is greater than a certain threshold, it can be considered that the two graphs are similar, and F1 and F2 intersect at this time. At this time, the two files can be judged to be functionally similar. It can be analyzed that the code contains redundant or unreachable statements, and the order of the statements in the code is adjusted, but the code completes the same function and other similar situations.

9. The method of claim 7, wherein, The content E2 specifically includes the content: E2.1, first determine whether the maximum common subgraph H is isomorphic to G1; E2.2, then determine whether the maximum common subgraph H is isomorphic to G2; if H is isomorphic to G1 and G2, it means that G1 and G2 are equivalent, and it can be determined that the file codes are similar; if H is isomorphic to G1 or G2, it means that G1 and G2 have an implication relationship, and it can be determined that the file codes are similar; E2.3, if the maximum common subgraph H is not isomorphic to G1 and G2, determine whether H is greater than a preset threshold; if it is greater than the preset threshold, it means that G1 and G2 have an intersection relationship, and it can be determined that the file codes are similar; if it is less than the preset threshold, it means that G1 and G2 do not have an intersection relationship, and it can be determined that the file codes are not similar; E2.5, end.

Images