A code clone detection method, device and storage medium

Abstract

The present invention provides a code clone detection method, device and storage medium. The code clone detection method includes: obtaining code text; generating a control flow chart according to the execution order of code statements in the code text; determining the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow chart; calculating a hash value corresponding to the code text based on the determined weights of each code statement; and determining the clone relationship between the code texts according to the hash values ​​corresponding to each code text. The above code clone detection method determines the weight of the code statement so that the calculated hash value corresponding to the code text contains the importance of the code statement, thereby effectively improving the accuracy of code clone detection.

Description

Technical Field

[0001] The present invention relates to the technical field of code similarity detection, and in particular to a code clone detection method, device and storage medium. Background Art

[0002] Code cloning refers to copying a piece of code from a software directly or making a few modifications, and then using it in other software as part of the latter's code. There are many reasons for code cloning, the main source of which is cloning by developers to reduce workload, including copying and modifying existing code snippets, using ready-made development framework design patterns, etc. Code clone detection is an important technology in reverse engineering, which helps reverse analysts search for code snippets and analyze the properties of target code by identifying repeated or known code components. Code cloning is divided into syntactic cloning and semantic cloning. Most of the technologies for detecting syntactic cloning currently use methods such as the frequency of words in the code text, the similarity of code blocks, and keyword matching to determine whether it is a clone. The various existing code clone detection tools are insufficient in terms of efficiency and other effects in large-scale code clone detection. Summary of the invention

[0003] In view of the above technical problems existing in the prior art, the present invention provides a code clone detection method, device and storage medium, which can determine the weight of the code statement so that the calculated hash value contains the importance of the code statement, thereby improving the accuracy of code clone detection.

[0004] An embodiment of the present invention provides a method for detecting code cloning, comprising:

[0005] Get the code text;

[0006] Generate a control flow graph according to the execution order of the code statements in the code text;

[0007] Determining a weight of a target code statement based on a degree of association between the target code statement and other code statements in the control flow graph;

[0008] Calculating a hash value corresponding to the code text based on the determined weights of each code statement;

[0009] The clone relationship between the code texts is determined according to the hash value corresponding to each code text.

[0010] In some embodiments, determining the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow graph specifically includes:

[0011] Generating a plurality of independent control flow sequences according to the execution paths represented by the control flow graph, wherein the control flow sequence includes at least one code statement;

[0012] The weight of the target code statement is determined based on the degree of association between the target code statement and other code statements in the control flow sequence.

[0013] In some embodiments, the calculating of the hash value corresponding to the code text based on the determined weights of each code statement specifically includes:

[0014] Calculate the corresponding statement encoding according to the weight of the code statement;

[0015] determining an overall encoding of all code statements based on the calculated statement encodings;

[0016] The overall code is binarized to obtain a hash value.

[0017] In some embodiments, calculating the corresponding statement code according to the weight of the code statement specifically includes:

[0018] The encoding of at least one post-position keyword in each code statement is averaged by column to obtain a post-position encoding;

[0019] The code corresponding to at least one front keyword located before the rear keyword is concatenated with the rear code in rows to obtain a statement code corresponding to the code statement.

[0020] In some embodiments, the generating of a plurality of independent control flow sequences according to the execution path represented by the control flow graph specifically includes:

[0021] Determine the entry address contained in the code text;

[0022] Determine, based on the entry address and a plurality of code statements included in the code text, a first statement and a second statement including a calling relationship, wherein the second statement is a called statement;

[0023] At least one control flow sequence including an exit and an entry is determined according to the code statement having a branch instruction, wherein the control flow sequence includes a second statement that has been called.

[0024] In some embodiments, determining the clone relationship between the code texts according to the hash values ​​corresponding to the code texts specifically includes:

[0025] Determining a Hamming distance between the code texts based on the hash values ​​of the code texts;

[0026] When the Hamming distance is less than a preset threshold, it is determined that there is a clone relationship between the corresponding code texts.

[0027] In some embodiments, after obtaining the code text, the method further includes:

[0028] A plurality of keywords in a code statement of a code text is intercepted to obtain a preprocessed code text.

[0029] The embodiment of the present invention further provides a code cloning detection device, comprising:

[0030] An acquisition module configured to acquire code text;

[0031] A processing module configured to generate a control flow graph according to the execution order of the code statements in the code text;

[0032] Determining a weight of a target code statement based on a degree of association between the target code statement and other code statements in the control flow graph;

[0033] Calculating a hash value corresponding to the code text based on the determined weights of each code statement;

[0034] The clone relationship between the code texts is determined according to the hash value corresponding to each code text.

[0035] In some embodiments, the processing module is further configured to:

[0036] Generating a plurality of independent control flow sequences according to the execution paths represented by the control flow graph, wherein the control flow sequence includes at least one code statement;

[0037] The weight of the target code statement is determined based on the degree of association between the target code statement and other code statements in the control flow sequence.

[0038] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program implements the steps of the above-mentioned code clone detection method when executed by a processor.

[0039] Compared with the prior art, the beneficial effect of the embodiments of the present invention lies in that: the present invention determines the weight of the target code statement by the degree of association between the target code statement and other code statements in the generated control flow graph, and through the determined weight of the code statement, the calculated hash value corresponding to the code text includes the importance of the code statement, thereby effectively improving the accuracy of code clone detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with letter suffixes or different letter suffixes may represent different instances of similar parts. The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the specification and claims, are used to illustrate the disclosed embodiments. When appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0041] Figure 1 A flowchart of a method for detecting code cloning according to an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of preprocessing of code text by a code clone detection method according to an embodiment of the present invention;

[0043] Figure 3 Another flow chart of the method for detecting code cloning according to an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of generating a control flow chart in a code cloning detection method according to an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of generating a control flow sequence in a code cloning detection method according to an embodiment of the present invention;

[0046] Figure 6 A calculation process diagram of calculation statement encoding in the code cloning detection method according to an embodiment of the present invention;

[0047] Figure 7 The structure block diagram of the code cloning detection device according to the embodiment of the present invention. DETAILED DESCRIPTION

[0048] Various aspects and features of the present invention are described herein with reference to the accompanying drawings.

[0049] It should be understood that various modifications may be made to the embodiments of the invention herein. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the invention will occur to those skilled in the art.

[0050] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the general description of the invention given above and the detailed description of the embodiments given below, serve to explain the principles of the invention.

[0051] These and other characteristics of the invention will become apparent from the following description of a preferred form of embodiment given as a non-limiting example, with reference to the accompanying drawings.

[0052] It should also be understood that although the invention has been described with reference to certain specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the invention.

[0053] The above and other aspects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0054] Specific embodiments of the present invention are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments invented are merely examples of the present invention, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present invention. Therefore, the specific structural and functional details invented herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present invention in a variety of ways with substantially any suitable detailed structure.

[0055] This specification may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," which may all refer to one or more of the same or different embodiments according to the present invention.

[0056] Simhash is a hash algorithm that can calculate text similarity. It selects keywords in the text based on TF-IDF (Termfrequency–inverse document frequency; weighted technology), uses a hash algorithm to binary encode the keywords, and then weights the encodings of multiple keywords to obtain a fixed-length encoding. Simhash is locally sensitive and can compress and reduce the original text content into a low-dimensional vector. In addition, the Simhash values ​​corresponding to texts with similar content are also relatively close.

[0057] The embodiment of the present invention provides a method for detecting code clones, such as Figure 1 As shown, the code clone detection method includes steps S101 to S105. The code clone detection method proposes an improvement on the Simhash algorithm, and the detection method is described below.

[0058] Step S101: Obtain code text.

[0059] Step S102: Generate a control flow chart according to the execution order of the code statements in the code text.

[0060] Step S103: Determine the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow graph.

[0061] Step S104: Calculate a hash value corresponding to the code text based on the determined weights of each code statement.

[0062] Step S105: Determine the cloning relationship between the code texts according to the hash value corresponding to each code text.

[0063] The above code text can be understood as a semi-structured special text, which can be applied to the Simhash algorithm. The code text includes multiple code statements that are executed in sequence along its execution path.

[0064] Specifically, a control flow chart can be used to characterize the execution logic of all code statements in a code text, wherein all code statements may be target code statements, that is, the embodiment of the present application is to determine the weights corresponding to all code statements respectively. For example, when determining the weight of one of all code statements, the code statement is determined as the target code statement, and the target code statement can be confirmed by the degree of association between the target code statement and other code statements. In this way, the weights corresponding to all code statements respectively are confirmed, that is, the importance of each code statement in the code text.

[0065] Specifically, the above-mentioned correlation degree can be understood as the correlation between each code statement and other code statements in each execution path of the code text, such as the number of executions, execution time, execution efficiency, etc. of the target code statement in the execution path. If the target code statement is executed more times, it can be understood that the code statement is more important than the code statement with fewer occurrences; if the target code statement is executed less times, it can be understood that the code statement is not as important as the code statement with more occurrences. In this way, in the subsequent process of determining the hash value based on the code statement, the hash value includes the information of the importance of each code statement in the code text, so as to provide a more reliable result for judging the clone relationship between code texts based on the hash value.

[0066] Specifically, the hash value corresponding to the code text can be understood as a fixed-length code. The hash value and the code text have a corresponding relationship. In the case of determining the clone relationship between two code texts, the hash values ​​corresponding to the two code texts can be compared to determine the clone relationship between the two code texts. Specifically, the comparison result of the two hash values ​​can be determined by comparing the distance, similarity and other parameters between the hash values.

[0067] The present invention determines the weight of the target code statement by the degree of association between the target code statement and other code statements in the generated control flow graph, and through the determined weight of the code statement, the calculated hash value corresponding to the code text includes the importance of the code statement, thereby effectively improving the accuracy of code clone detection.

[0068] In some embodiments, after obtaining the code text in step S101 and before step S102, the code clone detection method may further include: intercepting a plurality of keywords in the code statements of the code text to obtain preprocessed code text.

[0069] Specifically, the code text contains some special symbols, such as "[", "*", etc., and the semantics are more complex, including structures such as operators, operands, and control flows. The above preprocessing mainly converts the code text into a standard text, that is, removes special characters, retains only keywords such as instruction codes, operators, and operands, and intercepts the first three keywords of each code statement. If the code statement contains less than three keywords, "Null" (null pointer) can be added.

[0070] Specific combination Figure 2 , Figure 2 Schematic diagram of preprocessing of code text by the code clone detection method according to an embodiment of the present invention, Figure 2 The left side shows the obtained code text without preprocessing, and the right side shows the preprocessed code text after removing special characters and intercepting the first three keywords. This can reduce the processing amount of the code text and improve the detection speed.

[0071] In some embodiments, Figure 3 As shown, step S103: determining the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow graph, specifically including steps S201 to S202.

[0072] Step S201: Generate a plurality of independent control flow sequences according to the execution path represented by the control flow graph, wherein the control flow sequence includes at least one code statement.

[0073] Step S202: Determine the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow sequence.

[0074] Specifically, combined Figure 4 , Figure 4This is a schematic diagram of generating a control flow chart in the code cloning detection method of an embodiment of the present invention. The multiple code statements after preprocessing can be divided into a basic block. There is execution logic between the code statements in each basic block. The above-mentioned control flow chart can be generated between the multiple basic blocks contained in the code text according to the execution order.

[0075] Specifically, combined Figure 5 , Figure 5 This is a schematic diagram of generating a control flow sequence in the code cloning detection method of an embodiment of the present invention. After the control flow graph is generated, each branch generates an independent control flow sequence, and then weights are assigned to the code statements in the basic block according to the number of times each basic block appears in each control flow sequence.

[0076] Specifically, Figure 5 As shown, based on Figure 4 The control flow graph shown in FIG. 4 can generate five control flow sequences, and each basic block in the control flow sequence is assigned a unique block number (such as Figure 5 The digital encoding on each basic block shown in ), and then determine the number of times each basic block appears in all control flow sequences, such as Figure 5 As shown, basic block No. 1 appears in 5 control flow sequences, so the weight of the 5 code statements contained in basic block No. 1 is 5; basic block No. 2 only appears in 1 control flow sequence, so the weight of the 3 code statements contained in basic block No. 2 is 1, and so on, the weight of each code statement can be obtained.

[0077] In some embodiments, step S104: calculating a hash value corresponding to the code text based on the determined weights of each code statement, specifically includes:

[0078] Calculate the corresponding statement encoding according to the weight of the code statement;

[0079] determining an overall encoding of all code statements based on the calculated statement encodings;

[0080] The overall code is binarized to obtain a hash value.

[0081] Specifically, before calculating the statement code of each code statement, multiple keywords in the code statement with determined weights may be hashed to obtain three hash values ​​corresponding to the code statement, and the statement code is determined based on the hash values ​​and weights. This application does not specifically limit the number of the above keywords, and the following description takes three keywords as an example. Of course, the number of keywords may also be two, four, etc.

[0082] Specific combination Figure 6 , Figure 6FIG. 1 is a calculation process diagram of the calculation statement encoding in the code cloning detection method according to an embodiment of the present invention, Figure 6 In the example of the code statement "mov qword ptr" in basic block No. 1, the three keywords of the basic block can be hashed to obtain hash values ​​of "001111", "110101", and "101010". Since the weight of the code statement in basic block No. 1 is determined to be 5, the hash value can be assigned a weight, and the statement encoding corresponding to the statement can be obtained after the weight is assigned.

[0083] Specifically, after calculating and obtaining the sentence code of each code sentence, the overall code of all code sentences can be obtained by accumulating them in columns, thereby determining the overall code uniquely corresponding to each code text.

[0084] Specifically, after obtaining the overall code, the overall code can be binarized, wherein the binarization process is to transform the original code into a Simhash value of only 0 and 1. For example, after accumulation, the overall code "10 -3 -4 12 7 5" is obtained, and after binarization, the hash value of "1 0 0 1 1 1" is obtained.

[0085] In some embodiments, calculating the corresponding statement code according to the weight of the code statement specifically includes:

[0086] The encoding of at least one post-position keyword in each code statement is averaged by column to obtain a post-position encoding;

[0087] The code corresponding to at least one front keyword located before the rear keyword is concatenated with the rear code in rows to obtain a statement code corresponding to the code statement.

[0088] Specifically, each code statement includes at least two keywords. Figure 2 As shown in the figure, the present application proposes that each code statement contains three keywords. This is only an example, and the present application does not specifically limit the number of keywords. When there are three keywords in the code statement, there is a front-to-back relationship between the three keywords. The keyword at the front can be understood as the above-mentioned front keyword, and the keyword at the back can be understood as the back keyword.

[0089] Continue to combine Figure 6 , Figure 6 The "001111" shown in the figure is the hash value corresponding to the front keyword, and the "110101" and "101010" shown in the figure can be understood as the hash value corresponding to the rear keyword. Of course, this application does not specifically limit the number of front keywords and rear keywords. When there are 3 keywords, the first two digits can also be the front keywords, and the last two digits can be the rear keywords.

[0090] Further, continue to combine Figure 6 After assigning the weights to obtain the statement encoding corresponding to the statement, the last two keywords can be used as the post-position keywords and averaged by column to obtain the post-position encoding. Then, the encoding corresponding to the front keyword and the post-position encoding can be concatenated by row to obtain the statement encoding of "mov qword ptr" of basic block No. 1.

[0091] In some embodiments, step S201: generating a plurality of independent control flow sequences according to the execution path represented by the control flow graph specifically includes:

[0092] Determine the entry address contained in the code text;

[0093] Determine, based on the entry address and a plurality of code statements included in the code text, a first statement and a second statement including a calling relationship, wherein the second statement is a called statement;

[0094] At least one control flow sequence including an exit and an entry is determined according to the code statement having a branch instruction, wherein the control flow sequence includes a second statement that has been called.

[0095] Specifically, after determining the entry address of the code text, the code statements in each code text can be executed in sequence according to the execution logic of the code text, wherein some code statements have a calling relationship with other code statements, such as code statement A is the entry address, and the code text includes code statement A, code statement B and code statement E in sequence, wherein code statement A is the entry address, code statement B has code statement C and code statement D that have a calling relationship with it and are executed in sequence (code statement B can be understood as the first statement, and code statement C and code statement D can be understood as the second statement), and code statement E can have transfer instructions, such as call, jmp, ret, je, etc. At this time, a control flow sequence can be generated based on code statement A, code statement C, code statement D and code statement E, and so on, based on this method, multiple control flow sequences including an exit and an entry can be determined.

[0096] In some embodiments, step S105: determining the clone relationship between the code texts according to the hash values ​​corresponding to the code texts, specifically includes:

[0097] Determining a Hamming distance between the code texts based on the hash values ​​of the code texts;

[0098] When the Hamming distance is less than a preset threshold, it is determined that there is a clone relationship between the corresponding code texts.

[0099] Specifically, the Hamming distance between two strings of equal length is the number of different characters in corresponding positions of the two strings. For example, if the hash values ​​corresponding to two code texts are "1 1 0 1 1 0" and "1 1 0 0 1 0" respectively, it can be determined that the Hamming distance between the two code texts is 1.

[0100] Specifically, the present application does not make any specific limitation on the preset threshold of the Hamming distance. It can be obtained based on experience and multiple experiments that when the Hamming distance between two code texts exceeds the preset threshold, it indicates that there is no clone relationship between the two code texts; when the Hamming distance between the two code texts does not exceed the preset threshold, it indicates that there is a clone relationship between the two code texts.

[0101] Exemplarily, the preset threshold of the above-mentioned Hamming distance may be 3. When the hash values ​​corresponding to the two code texts are "1 1 0 1 1 0" and "1 1 0 0 1 0" respectively, it can be determined that the Hamming distance of the above-mentioned two code texts is 1, which is less than the preset threshold value 3, that is, the two code texts have a clone relationship; when the hash values ​​corresponding to the two code texts are "0 0 0 11 1" and "1 1 0 0 1 0" respectively, it can be determined that the Hamming distance of the above-mentioned two code texts is 4, which is greater than the preset threshold value 3, that is, the two code texts do not have a clone relationship.

[0102] The embodiment of the present invention further provides a code cloning detection device 110, such as Figure 7 As shown, the code clone detection device 110 includes an acquisition module 101 and a processing module 102. The acquisition module 101 is configured to acquire a code text. The processing module 102 is configured to generate a control flow graph according to the execution order of code statements in the code text; determine the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow graph; calculate the hash value corresponding to the code text based on the determined weights of each code statement; and determine the clone relationship between the code texts according to the hash values ​​corresponding to each code text.

[0103] The above code text can be understood as a semi-structured special text, which can be applied to the Simhash algorithm. The code text includes multiple code statements that are executed in sequence along its execution path.

[0104] Specifically, a control flow chart can be used to characterize the execution logic of all code statements in a code text, wherein all code statements may be target code statements, that is, the embodiment of the present application is to determine the weights corresponding to all code statements respectively. For example, when determining the weight of one of all code statements, the code statement is determined as the target code statement, and the target code statement can be confirmed by the degree of association between the target code statement and other code statements. In this way, the weights corresponding to all code statements respectively are confirmed, that is, the importance of each code statement in the code text.

[0105] Specifically, the above-mentioned correlation degree can be understood as the correlation between each code statement and other code statements in each execution path of the code text, such as the number of executions, execution time, execution efficiency, etc. of the target code statement in the execution path. If the target code statement is executed more times, it can be understood that the code statement is more important than the code statement with fewer occurrences; if the target code statement is executed less times, it can be understood that the code statement is not as important as the code statement with more occurrences. In this way, in the subsequent process of determining the hash value based on the code statement, the hash value includes the information of the importance of each code statement in the code text, so as to provide a more reliable result for judging the clone relationship between code texts based on the hash value.

[0106] Specifically, the hash value corresponding to the code text can be understood as a fixed-length code. The hash value and the code text have a corresponding relationship. In the case of determining the clone relationship between two code texts, the hash values ​​corresponding to the two code texts can be compared to determine the clone relationship between the two code texts. Specifically, the comparison result of the two hash values ​​can be determined by comparing the distance, similarity and other parameters between the hash values.

[0107] The present invention determines the weight of the target code statement by the degree of association between the target code statement and other code statements in the generated control flow graph, and through the determined weight of the code statement, the calculated hash value corresponding to the code text includes the importance of the code statement, thereby effectively improving the accuracy of code clone detection.

[0108] In some embodiments, the processing module 102 is further configured to: generate a plurality of independent control flow sequences according to the execution path represented by the control flow graph, wherein the control flow sequence includes at least one code statement; and determine the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow sequence.

[0109] In some embodiments, the processing module 102 is further configured to: calculate the corresponding statement code according to the weight of the code statement; determine the overall code of all code statements based on the calculated statement code; and binarize the overall code to obtain a hash value.

[0110] In some embodiments, the processing module 102 is further configured to: average the encoding of at least one post-keyword in each code statement by column to obtain a post-code; and concatenate the encoding corresponding to at least one pre-keyword located before the post-keyword with the post-code by row to obtain a statement code corresponding to the code statement.

[0111] In some embodiments, the processing module 102 is further configured to: determine an entry address contained in the code text; determine a first statement and a second statement containing a calling relationship based on the entry address and multiple code statements contained in the code text, wherein the second statement is a called statement; and determine at least one control flow sequence containing an exit and an entry based on the code statement having a transfer instruction, wherein the control flow sequence contains a second statement that has been called.

[0112] In some embodiments, the processing module 102 is further configured to: determine the Hamming distance between the code texts based on the hash value of each code text; and determine that the corresponding code texts have a cloning relationship when the Hamming distance is less than a preset threshold.

[0113] In some embodiments, the processing module 102 is further configured to: intercept multiple keywords in the code sentences of the code text to obtain the preprocessed code text.

[0114] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program implements the steps of the above-mentioned code clone detection method when executed by a processor.

[0115] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0116] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0117] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0118] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0119] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0120] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. It should be noted that, in the various components of the system of the present application, the components are logically divided according to the functions to be implemented, but the present application is not limited thereto, and the various components may be re-divided or combined as needed, for example, some components may be combined into a single component, or some components may be further decomposed into more sub-components.

[0121] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in the system according to the embodiment of the present application. The present application can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form. In addition, the present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

[0122] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0123] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that does not require protection is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a specific disclosed embodiment. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

[0124] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for detecting code clones, characterized in that: include: Get the code text; Generate a control flow graph according to the execution order of the code statements in the code text, the control flow graph is used to represent the execution logic of all the code statements in the code text; Determining a weight of a target code statement based on a degree of association between the target code statement and other code statements in the control flow graph; Obtaining the overall encoding of all code statements based on the determined weights of the individual code statements to calculate a hash value corresponding to the code text; Determine the clone relationship between the code texts according to the hash values ​​corresponding to the code texts; The step of determining the weight of the target code statement based on the degree of association between the target code statement and other code statements in the control flow chart specifically includes: Generating a plurality of independent control flow sequences according to the execution paths represented by the control flow graph, wherein the control flow sequence includes at least one code statement; Determining a weight of a target code statement based on a degree of association between the target code statement and other code statements in the control flow sequence; The degree of association is the association between each code statement and other code statements in each execution path of the code text, including at least one of the following: the number of executions of the target code statement in the execution path, the execution time of the target code statement in the execution path, and the execution efficiency of the target code statement in the execution path.

2. The code clone detection method according to claim 1, characterized in that: The step of calculating a hash value corresponding to the code text based on the determined weights of each code statement specifically includes: Calculate the corresponding statement code according to the weight of the code statement; determining an overall encoding of all code statements based on the calculated statement encodings; The overall code is binarized to obtain a hash value.

3. The code clone detection method according to claim 2, characterized in that: The step of calculating the corresponding statement code according to the weight of the code statement specifically includes: The encoding of at least one post-position keyword in each code statement is averaged by column to obtain a post-position encoding; The code corresponding to at least one front keyword located before the rear keyword is concatenated with the rear code in rows to obtain a statement code corresponding to the code statement.

4. The method for detecting code clones according to claim 1, characterized in that: The generating of a plurality of independent control flow sequences according to the execution path represented by the control flow graph specifically includes: Determine the entry address contained in the code text; Determine, based on the entry address and a plurality of code statements included in the code text, a first statement and a second statement including a calling relationship, wherein the second statement is a called statement; At least one control flow sequence including an exit and an entry is determined according to the code statement having a branch instruction, wherein the control flow sequence includes a second statement that has been called.

5. The method for detecting code clones according to claim 1, characterized in that: The clone relationship between the code texts is determined based on the hash values ​​corresponding to each code text, including: Determining a Hamming distance between the code texts based on the hash values ​​of the code texts; When the Hamming distance is less than a preset threshold, it is determined that there is a clone relationship between the corresponding code texts.

6. The method for detecting code clones according to claim 1, characterized in that: After obtaining the code text, the method further includes: A plurality of keywords in a code statement of a code text is intercepted to obtain a preprocessed code text.

7. A code clone detection device, characterized in that: include: An acquisition module configured to acquire code text; A processing module, configured to generate a control flow graph according to the execution order of the code statements in the code text, wherein the control flow graph is used to represent the execution logic of all the code statements in the code text; Determining a weight of a target code statement based on a degree of association between the target code statement and other code statements in the control flow graph; Obtaining the overall encoding of all code statements based on the determined weights of the individual code statements to calculate a hash value corresponding to the code text; Determine the clone relationship between the code texts according to the hash values ​​corresponding to the code texts; The processing module is further configured to: Generating a plurality of independent control flow sequences according to the execution paths represented by the control flow graph, wherein the control flow sequence includes at least one code statement; Determining a weight of a target code statement based on a degree of association between the target code statement and other code statements in the control flow sequence; The degree of association is the association between each code statement and other code statements in each execution path of the code text, including at least one of the following: the number of executions of the target code statement in the execution path, the execution time of the target code statement in the execution path, and the execution efficiency of the target code statement in the execution path.

8. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the code clone detection method according to any one of claims 1 to 6 are implemented.

Images