Distributed code clone detection and search method based on sub-block filtering, and system and medium

Abstract

The invention discloses a distributed code clone detection and a search method based on sub-block filtering and a system and a medium. The method comprises the steps that user codes and code library source codes are grouped respectively and preprocessed in parallel to obtain an intermediate file containing Token and frequency information of the Token; a global Token frequency table is establishedaccording to all the intermediate files; the code blocks of the code library source code are subjected to grouping and processing in parallel by using a global Token frequency table, and an index is established to obtain an index library; a keyword query index library is extracted from the user code by utilizing a sub-block filtering mechanism to obtain a candidate set corresponding to a code block of the user code; and for each code block in the user code, the similarity between the code block and each candidate code block in the corresponding candidate set is calculated, and if the similarity exceeds a preset value, the result that the code block is a clone pair is determined. The method has the characteristic of language irrelevance detection, is high in detection and search efficiency,is suitable for clone detection and search of large-scale codes, and supports a user to inquire the codes.

Description

technical field [0001] The present invention relates to the technical fields of distributed computing and code clone detection and search, in particular to a distributed code clone detection and search method, system and medium based on sub-block filtering. Background technique [0002] Code cloning refers to directly copying or slightly modifying a piece of code in one software, and then using it in other software as a component of the latter's code. There are many reasons for code cloning, the main source is the cloning performed by developers to reduce the workload, including copying and modifying existing code fragments, using ready-made development framework design patterns, etc. Code cloning is so common that between 5% and 20% of today's code snippets contain duplicated or slightly modified cloned code. Code reuse in high-quality open source software is more common, and more than 50% of the code has been reused. The use of open source components in various software ...

AI Technical Summary

Problems solved by technology

The current code search tools, such as Google Code Search, etc., can provide a low level of detection and cannot handle the retrieval and recommendation of similar codes

Images