Code cloning detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a code cloning detection method and device, electronic equipment and a storage medium, and relates to the field of computers. The method comprises the following steps: firstly, acquiring historical code files, constructing a historical code file set, and determining the similarity between code change contents of the historical code file set; according to the similarity, a part of historical code file sets are selected as clone code file sets, and computing power resources used for model training are saved; acquiring code change features corresponding to each historical code file in the cloned code file set; training a code clone detection model based on the marked clone code file set and code change features to obtain higher clone detection precision; and performing clone detection on the currently submitted code by using the trained code clone detection model so as to determine a clone detection result, thereby realizing code clone detection of submission-level granularity, improving software quality and reducing software maintenance cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, and in particular to a code clone detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Code clone refers to the occurrence of two or more pieces of code that are highly similar or even identical in function, structure or implementation during the software development process. For example, during the software development process, developers copy, paste and modify code in order to improve development efficiency, which can result in code clone. Although code clone can improve development efficiency in the short term, it can increase software code redundancy and security vulnerabilities in the long term, increasing software maintenance costs.

[0003] Therefore, there is a need to provide a code clone detection method to improve software quality and reduce maintenance costs. SUMMARY

[0004] The purpose of the present application is to provide a code clone detection method, device, electronic equipment and storage medium, which can improve code clone detection accuracy and range, thereby improving software quality and reducing software maintenance costs.

[0005] Therefore, the present application provides a code clone detection method, which comprises:

[0006] Obtaining each historical code file in a historical code repository and corresponding code change content;

[0007] Determining the similarity between the code change content of each historical code file in a set of historical code files; the set of historical code files includes any two or more historical code files;

[0008] Selecting a set of clone code files from each set of historical code files according to the similarity;

[0009] Obtaining the code change features corresponding to each historical code file in the set of clone code files;

[0010] Training a pre-constructed code clone detection model based on the labeled set of clone code files and the code change features until a predetermined training end condition is met;

[0011] After receiving the current submission code, performing code clone detection on the current submission code using the code clone detection model to determine the clone detection result of the current submission code.

[0012] Optionally, the similarity between the code change content of each historical code file in the set of historical code files comprises:

[0013] preprocessing the code change content; the preprocessing at least includes word segmentation processing;

[0014] inputting the preprocessed code change content into a target model constructed in advance to obtain an embedding representation vector generated by the target model;

[0015] determining vector similarity between embedding representation vectors of each of the historical code files;

[0016] determining string similarity between each of the historical code files;

[0017] taking the vector similarity and the string similarity as the similarity.

[0018] Optionally, the selecting a cloned code file set from each of the historical code file sets according to the similarity comprises:

[0019] obtaining a first preset range and a second preset range; the first preset range is a range corresponding to the vector similarity when there is code cloning between historical code files in the historical code file set; the second preset range is a range corresponding to the string similarity when there is code cloning between historical code files in the historical code file set;

[0020] if the vector similarity is in the first preset range or the string similarity is in the second preset range, the historical code file set is added to the cloned code file set.

[0021] Optionally, the obtaining the code change feature of each historical code file in the cloned code file set comprises:

[0022] obtaining a mapping relationship constructed in advance; the mapping relationship is used to represent the corresponding relationship between the code file and the identifier; the identifier is generated according to the hash value, the file name and the file path of the code file;

[0023] determining a target identifier of each of the historical code files in the cloned code file set according to the mapping relationship;

[0024] querying the code change feature of the historical code file corresponding to the target identifier from a code change feature library constructed in advance; the code change feature library is used to store the code change feature of each of the historical code files.

[0025] Optionally, the code change feature comprises at least one of a code change type, a code change line number, a code change time, a file name, code commit user information of each of the historical code files in the cloned code file set, and a similarity between any two of the historical code files in the cloned code file set, a difference between the code change line numbers, a difference between the code change times, a similarity between the file names.

[0026] The code change type comprises added code, deleted code and modified code, and the code change line number is a sum of an added code line number, a deleted code line number and a modified code line number.

[0027] Optionally, after receiving the current submitted code, the code clone detection model is used to perform code clone detection on the current submitted code to determine a clone detection result of the current submitted code.

[0028] The code change feature of each current code file involved in the current submitted code is determined.

[0029] The code change feature of each specified code file in the specified code repository is determined.

[0030] The current code file and the specified code file are combined to form a code file pair to be detected.

[0031] The code file pair to be detected and the corresponding code change feature are input into the code clone detection model to obtain a file-level clone detection result of the code file pair to be detected.

[0032] The clone detection result of the current submitted code is determined based on the file-level clone detection result.

[0033] Optionally, the file-level clone detection result comprises a clone result indicating that code cloning exists between the code files to be detected, and the clone detection result of the current submitted code is determined based on the file-level clone detection result, comprising:

[0034] The current code file corresponding to the clone result is taken as a cloned code file.

[0035] The total number of clone results corresponding to the cloned code file is taken as a cloned file repetition number of the cloned code file.

[0036] The code change line number of the cloned code file is taken as a cloned code change line number.

[0037] A cloned code proportion is determined according to the cloned code change line number and a total code line number of the cloned code file.

[0038] Based on the number of duplicate cloned files and the proportion of cloned code, each cloned code file is subjected to specified processing to obtain the clone detection result of the currently submitted code; the specified processing includes at least one of filtering, sorting, and grouping.

[0039] This application also provides a code clone detection device, the device comprising:

[0040] The historical file retrieval module is used to retrieve each historical code file in the historical code repository and the corresponding code change content;

[0041] The similarity determination module is used to determine the similarity between the code changes of each historical code file in the historical code file set; the historical code file set includes any two or more historical code files.

[0042] The clone set selection module is used to select a clone code file set from each of the historical code file sets based on the similarity.

[0043] The change feature acquisition module is used to acquire the code change features corresponding to each historical code file in the cloned code file set;

[0044] The model training module is used to train the pre-built code clone detection model based on the annotated clone code file set and the code change features until the preset training termination conditions are met.

[0045] The clone detection module is used to perform code clone detection on the current commit code using the code clone detection model after receiving the current commit code, so as to determine the clone detection result of the current commit code.

[0046] This application also provides an electronic device, including:

[0047] Memory, used to store computer programs;

[0048] A processor, configured to implement the steps of any of the code clone detection methods described above when executing the computer program.

[0049] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described code clone detection methods.

[0050] The beneficial effects of this invention lie in providing a code clone detection method, apparatus, electronic device, and storage medium. First, historical code files are acquired, a historical code file set is constructed, and the similarity between the code changes within the historical code file set is determined. Based on the similarity, a portion of the historical code file set is selected as the clone code file set, saving computational resources used for model training. Code change features corresponding to each historical code file in the clone code file set are obtained. Based on the labeled clone code file set and the code change features, a code clone detection model is trained to achieve higher clone detection accuracy. The trained code clone detection model is then used to perform clone detection on the currently submitted code to determine the clone detection result, achieving submission-level granular code clone detection. This results in a wider clone detection coverage, improved software quality, and reduced software maintenance costs. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A first flowchart of a code clone detection method provided in this application;

[0053] Figure 2 A schematic diagram illustrating the principle of a code clone detection method provided in this application;

[0054] Figure 3 This is a schematic diagram of the structure of a code clone detection device provided in this application;

[0055] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0056] The core of this invention is to provide a code clone detection method, apparatus, electronic device, and storage medium, which can improve the accuracy and range of code clone detection, thereby improving software quality and reducing software maintenance costs.

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please refer to Figure 1 , Figure 1 A first flowchart of a code clone detection method provided in this application, the method comprising:

[0059] S1. Retrieve each historical code file in the historical code repository and the corresponding code change content.

[0060] The historical code repository is used to centrally store, manage, and maintain the software's historical code files, and records the code changes involved in specified commits for each historical code file. Code change operations include adding code, deleting code, modifying code, file-level changes, and metadata changes; the code changes are the specific code involved in the aforementioned code change operations.

[0061] S2. Determine the similarity between the code changes corresponding to each historical code file in the historical code file set; the historical code file set includes any two or more historical code files.

[0062] This application combines any two or more historical code files from a historical code repository into a historical code file set, providing a data foundation for subsequently building cloned code file sets. The number of historical code file sets and the number of historical code files included in each historical code file set are not specifically limited in this application; they can be set according to actual needs.

[0063] For each set of historical code files, the similarity between the code changes made to each historical code file is determined in order to identify whether code clones exist between the historical code files based on the similarity. The implementation method for determining the similarity between the code changes will be described in subsequent embodiments and will not be elaborated here.

[0064] S3. Based on similarity, select a set of clone code files from each set of historical code files.

[0065] This application determines whether code clones exist among historical code files in a historical code file set based on the similarity between code changes. The historical code file set containing clones is selected as the aforementioned cloned code file set, which is then used to train the code clone detection model. The specific implementation of selecting the cloned code file set based on similarity will be described in subsequent embodiments and will not be elaborated here.

[0066] Furthermore, conventional file-level or function-level code detection can only detect code clones between complete code blocks, while this application performs code clone detection based on the similarity between code changes, enabling the detection of small code segment repetitions. This finer-grained clone detection results in more accurate clone detection outcomes. Based on this, this application achieves two advantages: firstly, a more reliable set of cloned code files, improving the quality of models trained using this set; and secondly, a reduced number of cloned code files, saving computational resources used during model training.

[0067] In summary, selecting cloned code files from the historical code file set based on similarity can save computational resources used during model training while ensuring the quality of the cloned code file set.

[0068] S4. Obtain the code change characteristics corresponding to each historical code file in the cloned code file set.

[0069] Code change features are used to characterize, in a specified commit, the code structure changes (including but not limited to the node types of the code blocks involved in the code change in the AST (Abstract Syntax Tree), the code semantic changes (including but not limited to the number of key API calls, the frequency of business keywords, etc.), the change scale (including but not limited to the number of new lines of code, the number of deleted lines of code, and the total number of lines changed), and the scope of impact. Code change features can be set according to actual needs, and this application does not impose any special limitations on them.

[0070] As an optional implementation, the code change features specifically include, within the cloned code file set, the code change type, number of lines changed, code change time, filename, and code submission user information for each historical code file. The code change features also include, within the cloned code file set, the similarity between any two historical code files, the difference in the number of lines changed, the difference in code change time, and the similarity between filenames. The code change type includes added code, deleted code, and modified code. The number of lines changed is the sum of the number of lines added, deleted, and modified. Furthermore, the similarity between the two historical code files includes both string similarity and vector similarity to reflect the similarity between historical code files from both textual and semantic perspectives. The filename similarity is a string similarity determined based on the filename.

[0071] Code change characteristics can help determine whether code clones exist between code files. The more similar the code change characteristics, the higher the likelihood of code cloning between the code files. For example, if two code files have the same code change type, number of lines changed, and user information for code submission, and the code change times are close, it is possible that the same user copied and pasted the code within a short period of time, and these two code files may be code clones.

[0072] As can be seen, this application not only focuses on the similarity of code files at the code level, but also integrates code change features to provide multi-dimensional features for training the code clone detection model, thereby improving the model's accuracy in detecting code clones.

[0073] S5. Based on the annotated cloned code file set and code change features, train the pre-built code clone detection model until the preset training termination conditions are met.

[0074] This application annotates the aforementioned cloned code file set to obtain a cloned code file set containing label information, so that the clone detection model can use the annotated cloned code file set for supervised learning. For example, clone labels are added to the cloned code file set containing cloned code, and uncloned labels are added to the cloned code file set containing no cloned code.

[0075] This application also inputs code change features into the code clone detection model, providing multi-dimensional learning features to improve the model's accuracy in identifying cloned files. Specifically, the code clone detection model can simultaneously capture the behavioral patterns of a single code file change, as well as the relationship structure between changes in different code files. The change features of a single code file help the model understand the nature and commit pattern of each commit, while the change features between two code files enable the model to identify the similarities, differences, and temporal relationships between changes, allowing it to more accurately identify code duplication.

[0076] The code clone detection model described above is a pre-built model. This application does not impose any particular restrictions on the algorithm used in the code clone detection model, such as logistic regression, K-nearest neighbors, support vector machines, and Naive Bayes. The specific algorithm can be selected according to actual needs, as long as the code detection model generates a detection result representing the existence of code clones based on the input information (code files and code change features).

[0077] Of course, this application demonstrates through experiments that ensemble learning models (such as XGBoost, CatBoost, and Random Forest) achieve excellent results in clone detection, exhibiting good generalization and robustness in complex environments, proving their application value and potential for widespread adoption in real-world scenarios. Therefore, ensemble learning models can be selected as code clone detection models.

[0078] This application does not impose any special limitations on the aforementioned preset training termination conditions, such as using the aforementioned cloned code file set to perform a preset number of training iterations or the preset loss function converging.

[0079] S6. After receiving the current commit code, use the code clone detection model to perform code clone detection on the current commit code to determine the clone detection result of the current commit code.

[0080] Upon receiving the current code commit, a pre-trained code clone detection model is used to detect code clones in the current commit, quickly locating any newly appearing code clones. Furthermore, since a developer's code commit may involve multiple code files, this application performs code clone detection at the commit level on all code files involved in the current commit, covering a wider range of code clone detection.

[0081] The specific process of using the code clone detection model to detect code clones in the current code submission will be explained in subsequent embodiments, and will not be repeated here.

[0082] In summary, this application selects a set of cloned code files from various historical code file sets based on the similarity between code changes, ensuring the quality of the cloned code file set while saving computational resources used during model training. Simultaneously, it integrates the similarity of code files at the code level with code change characteristics, providing multi-dimensional features to train the code clone detection model, improving the model's accuracy in detecting code clones and ensuring the effectiveness of the clone detection results. Finally, the trained code clone detection model is used to perform code clone detection on the current submission, achieving submission-level granular code clone detection and expanding the scope of code clone detection. Based on the clone detection results, developers can focus on code with clones, comprehensively assess the reliability of the software system, improve software quality, and reduce software maintenance costs.

[0083] Based on the above embodiments:

[0084] As an optional implementation, determining the similarity between the code changes in each historical code file within a historical code file set includes:

[0085] Preprocess the code changes; preprocessing should include at least word segmentation.

[0086] The preprocessed code changes are input into the pre-built target model to obtain the embedded representation vector generated by the target model.

[0087] Determine the vector similarity between the embedding representation vectors of each historical code file;

[0088] Determine the string similarity between the aforementioned historical code files;

[0089] Vector similarity and string similarity are used as similarity scores.

[0090] In this embodiment, the code changes are first preprocessed. Preprocessing includes at least word segmentation to provide basic units for subsequent vectorization. Preprocessing may also include removing blank lines, removing comments, removing stop words, and normalization, which can be configured according to actual needs. The word segmentation process can dynamically generate sub-word sequences using a word segmenter (e.g., a Unigram segmenter), such as segmenting "setFeedbacktype" into "set", "Feedback", and "type". This application does not impose any particular limitations on this.

[0091] The target model described above is used to process the input code changes and obtain the corresponding embedding representation vector. The embedding representation vector is used to represent the semantic information of the code changes. This application does not impose any particular limitation on the specific implementation of the target model; any model capable of mapping the code changes to a dense vector representing its semantic information (i.e., the aforementioned embedding representation vector) is acceptable. For example, the target model can be a Transformer-based encoder or encoder-decoder architecture, such as CodeBERT, PLBART, and CodeT5. This application also does not impose any particular limitation on the specific implementation of the target model in determining the embedding representation vector; for example, the average value of the hidden states of the last layer of the model can be used as the embedding representation vector, or a weighted combination of the hidden states of several intermediate layers of the model can be used.

[0092] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the principle of a code clone detection method provided in this application. After determining the embedding representation vectors of each historical code file in the historical code file set, the vector similarity between each embedding representation vector is determined. This embodiment also determines the string similarity between each historical code file. Vector similarity can evaluate the similarity between the code changes in each historical code file from a semantic similarity perspective. String similarity is used to determine whether the code in the code changes is similar from a text similarity perspective, accurately reflecting whether the code changes are clones that are "copy-paste-slightly modified".

[0093] In this embodiment, multiple methods can be used simultaneously to determine the vector similarity between embedded representation vectors. As an optional embodiment, determining the vector similarity between embedded representation vectors of a historical code file set includes: determining the cosine similarity, Manhattan distance, and Euclidean distance between the embedded representation vectors of the historical code file set. Cosine similarity helps determine whether the code changes in the historical code file set achieve similar functions (e.g., fixing the same type of vulnerability); Manhattan distance and Euclidean distance help determine whether the degree of code changes is similar.

[0094] The string similarity mentioned above includes, but is not limited to, edit distance similarity, Jaro-Winkler distance similarity, and longest common subsequence similarity. The specific type can be selected according to actual needs.

[0095] In summary, this embodiment comprehensively evaluates the similarity between historical code files in the historical code file set by integrating similarity from different dimensions (including semantic similarity and textual similarity), so as to comprehensively and accurately assess whether there are code clones in each historical code file in the historical code file set.

[0096] As an optional implementation, a set of cloned code files is selected from various historical code file sets based on similarity, including:

[0097] Obtain a first preset range and a second preset range; the first preset range is the range of vector similarity when there is code cloning between historical code files in the historical code file set; the second preset range is the range of string similarity when there is code cloning between historical code files in the historical code file set.

[0098] If the vector similarity is within the first preset range, or the string similarity is within the second preset range, then the historical code file set is added to the cloned code file set.

[0099] In the above embodiments, the similarity between historical code files in the historical code file set is evaluated from two dimensions: semantic similarity and textual similarity, resulting in vector similarity and string similarity between the historical code files. Based on this, this embodiment pre-sets a first preset range and a second preset range. The first preset range is the range of vector similarity when code clones exist between historical code files in the historical code file set; the second preset range is the range of string similarity when code clones exist between historical code files in the historical code file set. The specific values ​​of the similarity ranges can be set according to actual needs, and this application does not impose any particular limitations on them.

[0100] It should also be noted that if multiple calculation methods are used to determine vector similarity, a corresponding first preset range is set for each type of vector similarity. For example, a corresponding first preset range is set for cosine similarity, Manhattan distance, and Euclidean distance.

[0101] In this embodiment, as long as any similarity is within the corresponding similarity range, the historical code file set is added to the clone code file set, thereby achieving effective filtering of the historical code file set and obtaining a sufficient clone code file set to provide a data foundation for subsequent model training.

[0102] As an optional implementation, a clone code file set is selected from each historical code file set based on similarity, including: determining a first similarity based on a preset first weight and vector similarity; determining a second similarity based on a preset second weight and string similarity; determining a comprehensive similarity based on the first and second similarities; and adding historical code file sets with a comprehensive similarity greater than a preset similarity threshold to the clone code file set. This method can effectively eliminate file sets with low code cloning probability from a massive historical code file set, narrowing the scope and obtaining a high-quality clone code file set, further saving the computing resources required for model training.

[0103] As an optional embodiment, obtaining the code change characteristics of each historical code file in the cloned code file set includes:

[0104] Obtain the pre-built mapping relationship; the mapping relationship is used to represent the correspondence between code files and identifiers; the identifier is generated based on the hash value, file name, and file path of the code file;

[0105] Based on the mapping relationship, determine the target identifier of each historical code file in the cloned code file set;

[0106] From the pre-built code change feature library, query the code change features of the historical code file corresponding to the target identifier; the code change feature library is used to store the code change features of each historical code file.

[0107] In this embodiment, a mapping relationship is pre-built to represent the correspondence between code files and identifiers. Each code file corresponds to a unique identifier, which can also be understood as a unique key. The identifier can be generated based on the hash value, filename, and file path of the code file. Generating an identifier by combining the filename and file path facilitates finding the corresponding code file based on the identifier.

[0108] Based on this, the target identifier of each historical code file in the cloned code file set is determined according to the mapping relationship; then, the code change features of the historical code file corresponding to the target identifier are queried from the pre-built code change feature library; the code change feature library is used to store the code change features of each historical code file.

[0109] The aforementioned code change characteristics include the code change type, number of lines changed, code change time, filename, and code submission user information for each historical code file in the cloned code file set. The code change characteristics also include the similarity between any two historical code files in the cloned code file set, the difference in the number of lines changed, the difference in code change time, and the similarity between filenames. The code change type includes added code, deleted code, and modified code. The number of lines changed is the sum of the number of lines added, deleted, and modified. Furthermore, the similarity between two historical code files includes both string similarity and vector similarity to reflect the similarity between historical code files from both textual and semantic perspectives. The filename similarity is a string similarity determined based on the filename.

[0110] In summary, in this embodiment, the target identifier corresponding to the historical code file is determined based on the mapping relationship, and then the code change features corresponding to the historical code file are found through the target identifier, so as to achieve efficient and accurate search for code change features, which facilitates subsequent code clone detection based on code change features.

[0111] As an optional embodiment, upon receiving the current commit code, a code clone detection model is used to perform code clone detection on the current commit code to determine the clone detection result of the current commit code, including:

[0112] Identify the current code files involved in the current code commit, and the characteristics of the code changes in the current code files;

[0113] Determine the code change characteristics of each specified code file in a specified code repository;

[0114] Combine the current code file and the specified code file into a code file pair to be detected;

[0115] Input the code file pair to be detected and the corresponding code change features into the code clone detection model to obtain the file-level clone detection results of the code file pair to be detected;

[0116] Based on the clone detection results at each file level, determine the clone detection result of the currently submitted code.

[0117] Considering that the current code commit may involve multiple code files, upon receiving the current code commit, the first step is to identify each current code file involved. This embodiment detects whether there are code clones between each current code file and each specified code file in a specified code repository; therefore, the current code file and the specified code files are combined into a pair of code files to be detected.

[0118] Furthermore, as mentioned above, this application utilizes code change features to assist in determining whether code clones exist between code files. Therefore, it is also necessary to determine the code change features corresponding to each current code file and each specified code file in order to improve the accuracy of code clone detection.

[0119] Next, the code files to be detected and their code change characteristics are input into the code clone detection model so that the model can determine the file-level clone detection results for the code file pairs to be detected. Each file-level clone detection result indicates whether a code clone exists between the current code file and the specified code file in the corresponding code file pair to be detected. By combining the results of each file-level clone detection, the clone detection result of the current commit code can be determined, that is, whether there is a code clone in each current code file in the current commit code. This achieves commit-level code clone detection, covers a wider range of clone detection, and improves code clone detection efficiency.

[0120] As an optional embodiment, the file-level clone detection result includes clone results characterizing the existence of code clones between the code files to be detected; based on the clone detection results of each pair of code files to be detected, the clone detection result of the currently submitted code is determined, including:

[0121] Use the current code file corresponding to the cloned result as the cloned code file;

[0122] The total number of cloned results corresponding to the cloned code file is taken as the number of duplicate cloned files of the cloned code file;

[0123] Use the number of lines of code changed in the cloned code file as the number of lines of code changed in the cloned code;

[0124] The proportion of cloned code is determined based on the number of lines of code changed in the cloned code and the total number of lines of code in the cloned code file;

[0125] Based on the number of duplicate cloned files and the proportion of cloned code, each cloned code file is subjected to specified processing to obtain the clone detection result of the currently submitted code; the specified processing includes at least one of filtering, sorting and grouping.

[0126] As described in the above embodiments, the clone result in the file-level clone detection result indicates that there is a code clone between the detected code file pair, that is, there is a code clone between the current code file and the specified code file in the detected code file pair. Therefore, this embodiment takes the current code file corresponding to the clone result as the cloned code file; the number of duplicate cloned files of the cloned code file is determined by the total number of clone results corresponding to the cloned code file. The more duplicate cloned files there are, the more serious the code cloning situation of the current code file.

[0127] This embodiment also uses the number of lines of code changed in the cloned code file as the cloned code change count. The cloned code change count is the sum of the number of newly added lines of code, the number of deleted lines of code, and the number of modified lines of code. Based on the cloned code change count and the total number of lines of code in the cloned code file, the cloned code ratio is determined. The higher the cloned code ratio, the more severe the code cloning situation of the current code file.

[0128] like Figure 2 As shown, based on the clone detection results of the code file pairs to be detected, clone detection results at the submission granularity are generated and displayed. Specifically, according to the number of duplicate clone files and the proportion of cloned code, each cloned code file is filtered, sorted, and grouped according to specified processing to obtain the clone detection results of the current submission code. For example, a preset number of duplicates and a preset code proportion are set in advance; code clone files with a number of duplicates greater than the preset number of duplicates or a cloned code proportion greater than the preset code proportion are used as target code files. Based on the number of duplicate clone files or the cloned code proportion, each target code file is sorted and output to obtain the clone detection results of the current submission code, showing developers each current code file involved in code cloning and the degree of code cloning in the current submission code.

[0129] Of course, the specified code files corresponding to each target code file can also be displayed, that is, the specified code files that have code clones with the target code files can be displayed, so that developers can understand the cloning situation and maintain the software in a timely manner.

[0130] In summary, this embodiment utilizes a trained code clone detection model to perform code clone detection on the current code commit, achieving commit-level code clone detection and expanding the scope of code clone detection. Based on the clone detection results, developers can focus on code containing clones, enabling a comprehensive assessment of the software system's reliability, improving software quality, and reducing software maintenance costs.

[0131] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a code clone detection device provided in this application. The code clone detection device includes:

[0132] The historical file acquisition module 301 is used to acquire each historical code file in the historical code repository;

[0133] The similarity determination module 302 is used to determine the similarity between the code change content of each historical code file set; the historical code file set includes any two historical code files;

[0134] The clone set selection module 303 is used to select a clone code file set from each historical code file set based on similarity.

[0135] The change feature acquisition module 304 is used to acquire the code change features corresponding to each historical code file in the cloned code file set;

[0136] The model training module 305 is used to train the pre-built code clone detection model based on the annotated clone code file set and code change features until the preset training termination conditions are met.

[0137] The clone detection module 306 is used to perform code clone detection on the current commit code after receiving the current commit code, so as to determine the clone detection result of the current commit code.

[0138] For a detailed description of the code clone detection device provided in this application, please refer to the embodiments of the code clone detection method described above; further details will not be repeated here.

[0139] As an optional embodiment, the similarity determination module 302 includes:

[0140] The preprocessing module is used to preprocess the code changes; preprocessing includes at least word segmentation.

[0141] The vector acquisition module is used to input the preprocessed code changes into the pre-built target model to obtain the embedded representation vector generated by the target model;

[0142] The first similarity determination module is used to determine the vector similarity between the embedded representation vectors of each historical code file.

[0143] The second similarity determination module is used to determine the string similarity between various historical code files;

[0144] The similarity determination submodule is used to determine the similarity between vector similarity and string similarity.

[0145] As an optional embodiment, the clone set selection module 303 includes:

[0146] The preset range acquisition module is used to acquire a first preset range and a second preset range; the first preset range is the range of vector similarity when there is code cloning between historical code files in the historical code file set; the second preset range is the range of string similarity when there is code cloning between historical code files in the historical code file set.

[0147] The clone set selection submodule is used to add the historical code file set to the clone code file set if the vector similarity is within a first preset range or the string similarity is within a second preset range.

[0148] As an optional embodiment, the change feature acquisition module 304 includes:

[0149] The mapping relationship acquisition module is used to obtain pre-built mapping relationships; the mapping relationship is used to represent the correspondence between code files and identifiers; the identifier is generated based on the hash value, file name, and file path of the code file;

[0150] The target identifier acquisition module is used to determine the target identifier of each historical code file in the cloned code file set according to the mapping relationship;

[0151] The feature query module is used to query the code change features of historical code files corresponding to a target identifier from a pre-built code change feature library; the code change feature library is used to store the code change features of each historical code file.

[0152] As an optional embodiment, the clone detection module 306 includes:

[0153] The current information determination module is used to determine the current code files involved in the current code submission, as well as the code change characteristics of the current code files;

[0154] The specified feature determination module is used to determine the code change characteristics of each specified code file in a specified code repository;

[0155] The code-to-detection module is used to combine the current code file and the specified code file into a code-to-detection pair.

[0156] The current detection module is used to input the code file pair to be detected and the corresponding code change features into the code clone detection model in order to obtain the file-level clone detection results of the code file pair to be detected.

[0157] The current result determination module is used to determine the clone detection result of the currently submitted code based on the clone detection results of each file level.

[0158] As an optional embodiment, the file-level clone detection result includes clone results characterizing the existence of code clones between the code files to be detected; the current result determination module includes:

[0159] The clone file determination module is used to select the current code file corresponding to the clone result as the clone code file;

[0160] The clone count determination module is used to determine the total number of clone results corresponding to the clone code file as the number of duplicate clone files of the clone code file.

[0161] The clone line count determination module is used to determine the number of lines of code changes in the cloned code file as the cloned code change count.

[0162] The clone ratio determination module is used to determine the clone ratio based on the number of lines of code changed in the cloned code and the total number of lines of code in the cloned code file.

[0163] The current result determination submodule is used to perform specified processing on each cloned code file based on the number of duplicate cloned files and the proportion of cloned code, and obtain the clone detection result of the currently submitted code; the specified processing includes at least one of filtering, sorting and grouping.

[0164] Please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of an electronic device, which includes:

[0165] Memory 401 is used to store computer programs;

[0166] Processor 402 is used to implement the steps of any of the above code clone detection methods when executing a computer program.

[0167] For a detailed description of the electronic device provided in this application, please refer to the embodiments of the code clone detection method described above; this application will not repeat the details here.

[0168] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described code clone detection methods.

[0169] For a detailed description of the storage medium provided in this application, please refer to the embodiments of the code clone detection method described above; this application will not repeat the details here.

[0170] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. It should also be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus.

[0171] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A code clone detection method, characterized in that, The method includes: Retrieve each historical code file in the historical code repository, along with the corresponding code changes; Determine the similarity of code changes among historical code files in a historical code file set; the historical code file set includes any two or more historical code files. Based on the similarity, a set of cloned code files is selected from each of the historical code file sets; Obtain the code change characteristics corresponding to each historical code file in the cloned code file set; Based on the annotated set of cloned code files and the code change features, the pre-built code clone detection model is trained until the preset training termination condition is met. Upon receiving the current commit code, the code clone detection model is used to perform code clone detection on the current commit code to determine the clone detection result of the current commit code.

2. The code clone detection method as described in claim 1, characterized in that, The determination of the similarity between the code changes in each historical code file within the historical code file set includes: The code changes are preprocessed; the preprocessing includes at least word segmentation. The preprocessed code changes are input into a pre-built target model to obtain the embedded representation vector generated by the target model. Determine the vector similarity between the embedding representation vectors of each of the historical code files; Determine the string similarity between the aforementioned historical code files; The vector similarity and the string similarity are used as the similarity.

3. The code clone detection method as described in claim 2, characterized in that, The step of selecting a set of clone code files from each of the historical code file sets based on the similarity includes: Obtain a first preset range and a second preset range; the first preset range is the range corresponding to the vector similarity when code clones exist among historical code files in the historical code file set; the second preset range is the range corresponding to the string similarity when code clones exist among historical code files in the historical code file set. If the vector similarity is within the first preset range, or the string similarity is within the second preset range, then the historical code file set is added to the cloned code file set.

4. The code clone detection method as described in claim 1, characterized in that, The step of obtaining the code change characteristics of each historical code file in the cloned code file set includes: Obtain a pre-constructed mapping relationship; the mapping relationship is used to represent the correspondence between code files and identifiers; the identifier is generated based on the hash value, file name, and file path of the code file; Based on the mapping relationship, the target identifier of each of the historical code files in the cloned code file set is determined; From the pre-built code change feature library, query the code change features of the historical code files corresponding to the target identifier; the code change feature library is used to store the code change features of each of the historical code files.

5. The code clone detection method as described in claim 1, characterized in that, The code change features include the code change type, number of lines changed, code change time, file name, and code submission user information corresponding to each of the historical code files in the cloned code file set, as well as at least one of the following: similarity between any two historical code files in the cloned code file set, difference in the number of lines changed, difference in code change time, and similarity between file names. The code change types include adding code, deleting code, and modifying code; the number of lines of code changed is the sum of the number of lines of code added, deleted, and modified.

6. The code clone detection method as described in claim 1, characterized in that, The step of performing code clone detection on the current commit code using the code clone detection model after receiving the current commit code, in order to determine the clone detection result of the current commit code, includes: Identify each current code file involved in the current code commit, and the code change characteristics of the current code file; Determine the code change characteristics of each specified code file in a specified code repository; The current code file and the specified code file are combined to form a code file pair to be detected; The code file pair to be detected and the corresponding code change features are input into the code clone detection model to obtain the file-level clone detection results of the code file pair to be detected. Based on the file-level clone detection results, the clone detection result of the currently submitted code is determined.

7. The code clone detection method as described in claim 6, characterized in that, The file-level clone detection results include clone results that characterize the existence of code clones between the code files to be detected; The process of determining the clone detection result of the currently submitted code based on the file-level clone detection results includes: The current code file corresponding to the cloned result is used as the cloned code file; The total number of cloned results corresponding to the cloned code file is taken as the number of duplicate cloned files of the cloned code file; The number of lines of code changes in the cloned code file is taken as the number of lines of code changes in the cloned code. The proportion of cloned code is determined based on the number of lines of code changed in the cloned code and the total number of lines of code in the cloned code file; Based on the number of duplicate cloned files and the proportion of cloned code, each cloned code file is subjected to specified processing to obtain the clone detection result of the currently submitted code; the specified processing includes at least one of filtering, sorting, and grouping.

8. A code clone detection device, characterized in that, The device includes: The historical file retrieval module is used to retrieve each historical code file in the historical code repository and the corresponding code change content; The similarity determination module is used to determine the similarity between the code changes of each historical code file in the historical code file set; the historical code file set includes any two or more historical code files. The clone set selection module is used to select a clone code file set from each of the historical code file sets based on the similarity. The change feature acquisition module is used to acquire the code change features corresponding to each historical code file in the cloned code file set; The model training module is used to train the pre-built code clone detection model based on the annotated clone code file set and the code change features until the preset training termination conditions are met. The clone detection module is used to perform code clone detection on the current commit code using the code clone detection model after receiving the current commit code, so as to determine the clone detection result of the current commit code.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the code clone detection method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the code clone detection method as described in any one of claims 1 to 7.

Images