SCA-based code homology detection method and device, computer equipment and medium

Abstract

The invention provides an SCA-based code homology detection method and device, computer equipment and a medium, and the method comprises the steps: obtaining directory fingerprint information of a to-be-detected item; according to the directory fingerprint information of the to-be-detected item and directory fingerprint information of each open source item in a code fingerprint database, determining whether an association relationship exists between the to-be-detected item and each open source item; acquiring content fingerprint information of each code file in the to-be-detected item under the condition that any open source item does not have the incidence relation with the to-be-detected item; according to the directory information of the to-be-detected item and the content fingerprint information of each code file, screening a target open source item from each open source item; and according to the syntax tree fingerprint information, stored in the code fingerprint database, of each code file of the target open source project under the preset detection level and the syntax tree fingerprint information, stored in the code fingerprint database, of each code file of the to-be-detected project under the preset detection level, determining the homologous relationship between the to-be-detected project and the target open source project.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a code homology detection method, apparatus, computer equipment and medium based on SCA. Background Art

[0002] Code homology detection is also called homology analysis, which refers to the homology analysis of the components in an application or software. Code homology detection technology is mainly used to detect the same code components between a certain code fragment in the application source code and other code fragments or open source codes in the project.

[0003] In the related code homology detection technology, hash value matching is usually used to determine whether a certain code fragment in the application source code is homologous to other code fragments or open source code in the project. However, for projects where the code has changed but the syntax tree logic remains unchanged, using the above code homology detection method will produce detection results of different sources, resulting in inaccurate homology detection results. Therefore, for projects where the code has changed, how to obtain accurate homology detection results is a problem worthy of attention. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a code homology detection method, apparatus, computer equipment and medium based on SCA.

[0005] In a first aspect, the present disclosure provides a code homology detection method based on SCA, including: Get the directory fingerprint information of the project to be detected; Determine whether there is an association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library; In the case that there is no association relationship between any of the open source projects and the project to be detected, obtaining content fingerprint information of each code file in the project to be detected; According to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, a target open source project whose first similarity with the project to be detected is greater than a first set threshold is selected from each open source project; According to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level, and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, the homology relationship between the project to be detected and the target open source project is determined.

[0006] In an optional implementation manner, the code fingerprint library is determined according to the following steps: Obtain version information, directory information, and shared project information for each version of open source projects; For each version of the open source project, determine the target component to which the open source project of this version belongs based on the version information of the open source project of this version, the common project information, and the component information of each component stored; determine the correspondence between the open source project and the component based on the target component to which each version of the open source project belongs; For each version of the open source project, generate directory fingerprint information of the open source project of this version according to the directory information of the open source project of this version and the content fingerprint information of each code file contained in the open source project of this version; Extract syntax tree structure information from each code file contained in the open source project to obtain syntax tree structure information of each code file; For each code file, generate file-level syntax tree fingerprint information of the code file according to the syntax tree structure information of the code file; determine each function in the code file, and determine the function-level syntax tree fingerprint information of each function according to the syntax tree structure information of the code file; determine the node of each line in the code file, and determine the syntax tree fingerprint information of each node according to the syntax tree structure information of the code file, and determine the fragment-level syntax tree fingerprint information according to the syntax tree fingerprint information of each node; The correspondence between open source projects and components, the syntax tree structure information of each code file, the file-level syntax tree fingerprint information of each code file, the function-level syntax tree fingerprint information of each function in each code file, and the fragment-level syntax tree fingerprint information of each code file are stored in the code fingerprint library.

[0007] In an optional implementation, the file-level syntax tree fingerprint information of each code file, the function-level syntax tree fingerprint information of each function in each code file, and the fragment-level syntax tree fingerprint information of each code file are stored in a code fingerprint library, including: Generate content fingerprint information of each code file, and determine a set of code files with the same content fingerprint information; The file-level syntax tree fingerprint information, function-level syntax tree fingerprint information and fragment-level syntax tree fingerprint information of any code file in the code file set are stored in a code fingerprint library.

[0008] In an optional implementation, based on the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, a target open source project having a first similarity with the project to be detected greater than a first set threshold is selected from each open source project, including: Determine a fingerprint vector of the project to be detected according to the content fingerprint information of each code file in the project to be detected; the dimension of the fingerprint vector is the number of code files; the value of each element in the fingerprint vector is the content fingerprint information of the corresponding code file; Determine, according to the fingerprint vector of the project to be detected and the fingerprint vectors of each open source project, a preselected open source project whose distance from the project to be detected is less than a second set threshold; According to the directory information of the project to be detected and the local sensitive hash values ​​of each code file contained in the project to be detected, as well as the directory information of each pre-selected open source project and the local sensitive hash values ​​of each code file contained in each pre-selected open source project, a target open source project whose first similarity with the project to be detected is greater than a first set threshold is screened from the pre-selected open source projects.

[0009] In an optional implementation, determining the homology relationship between the project to be detected and the target open source project according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level includes: According to the syntax tree fingerprint information of each code file of the target open source project at a preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result; When the homology detection result indicates that the code file of the project to be detected and the code file of the target open source project are the same source code files, determining a second similarity between the project to be detected and the target open source project according to content fingerprint information of each of the same source code files; According to the second similarity, a homology relationship between the project to be detected and the target open source project is determined.

[0010] In an optional implementation, the preset detection level includes file-level detection, function-level detection or fragment-level detection; the detection accuracy of the fragment-level detection is greater than the detection accuracy of the function-level detection; the detection accuracy of the function-level detection is greater than the detection accuracy of the file-level detection; The syntax tree fingerprint information under the file level detection is file level syntax tree fingerprint information; The syntax tree fingerprint information under the function level detection is function level syntax tree fingerprint information; The syntax tree fingerprint information under the segment level detection is segment level syntax tree fingerprint information.

[0011] In an optional implementation, the preset detection level is file-level detection; According to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result, including: When the file-level syntax tree fingerprint information of any code file of the target open source project stored in the code fingerprint library is the same as the file-level syntax tree fingerprint information of any code file of the project to be detected, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code file.

[0012] In an optional implementation manner, the preset detection level is function-level detection or fragment-level detection; According to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result, including: According to the file-level syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library and the file-level syntax tree fingerprint information of each code file of the project to be detected, a homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a first homology detection result; When the first homology detection result indicates that the code files of the project to be detected are from different sources from the code files of the target open source project, a homology detection is performed on the code files of the project to be detected and the code files of the target open source project according to the syntax tree fingerprint information of the code files of the target open source project at a preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of the code files of the project to be detected at the preset detection level to obtain a second homology detection result.

[0013] In an optional implementation, when the preset detection level is function-level detection, according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at the preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a second homology detection result, including: If the number of valid lines of the longest common function-level syntax tree fingerprint information of any function in any code file of the project to be detected and any function in any code file of the target open source project is greater than the first preset number of valid lines, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code files.

[0014] In an optional implementation, when the preset detection level is fragment-level detection, according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at the preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a second homology detection result, including: If the number of valid lines of the longest common fragment-level syntax tree fingerprint information in any code file of the project to be detected and any code file of the target open source project is greater than the second preset number of valid lines, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code files.

[0015] In a second aspect, the embodiment of the present disclosure also provides a code homology detection device based on SCA, including: The first acquisition module is used to obtain directory fingerprint information of the item to be detected; A first determination module is used to determine whether there is an association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library; A second acquisition module is used to acquire content fingerprint information of each code file in the project to be detected when there is no association relationship between any of the open source projects and the project to be detected; a screening module, configured to screen, from each open source project, a target open source project whose first similarity with the project to be detected is greater than a first set threshold value, based on the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project; The second determination module is used to determine the homology relationship between the project to be detected and the target open source project based on the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level.

[0016] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0018] The SCA-based code homology detection method provided by the embodiment of the present disclosure can firstly make a preliminary judgment on the association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library. When it is determined that there is no association relationship between any open source project and the project to be detected, further, the open source project can be preliminarily screened according to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, to obtain a target open source project whose first similarity with the project to be detected is greater than a first set threshold value, and then the homology relationship between the project to be detected and the target open source project can be further determined according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at a preset detection level. According to the above homology detection process, the modified code files can also be accurately matched with homology open source projects, thereby improving the accuracy of code homology detection.

[0019] For a description of the effects of the above-mentioned SCA-based code homology detection device, computer equipment, and computer-readable storage medium, please refer to the description of the above-mentioned SCA-based code homology detection method, which will not be repeated here.

[0020] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of a code homology detection method in the related art is shown; Figure 2 A flow chart of a code homology detection method based on SCA provided in an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of the structure of a code homology detection device based on SCA provided in an embodiment of the present disclosure is shown; Figure 4 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0024] According to research, in the field of code homology detection, hash value matching is usually used to determine whether a certain code fragment in the application source code is homologous to other code fragments or open source code in the project. Figure 1As shown, first, open source projects and components are collected from the Internet; then, the open source projects and components are associated to obtain the associated components and projects; next, the associated components and projects are hashed to generate fingerprint data; finally, the fingerprint data is stored in the database. After obtaining the project to be detected, the fingerprint data of the project to be detected is also generated, and then the database is queried according to the fingerprint of the project to be detected. If the database contains the same fingerprint as the project to be detected, the open source project or component corresponding to the same fingerprint as the project to be detected is determined to be the open source project or component of the same source as the project to be detected; if the database does not contain the same fingerprint as the project to be detected, it is determined that the database does not contain the open source project or component of the code project to be detected.

[0025] However, for projects where the code has changed but the syntax tree logic remains unchanged, the above code homology detection method that only relies on hash value matching will produce detection results of different sources. In actual situations, changing the code that does not affect the syntax tree logic will not change the homology results of the project to be detected and other projects. Therefore, it is impossible to accurately obtain homology detection results by only using hash value matching. Therefore, for projects where the code has changed but the syntax tree logic remains unchanged, how to obtain accurate homology detection results is a problem worthy of attention.

[0026] Based on this, the present disclosure provides a code homology detection method based on SCA, which can first make a preliminary judgment on the association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library. When it is determined that there is no association relationship between any open source project and the project to be detected, the open source project can be preliminarily screened according to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, to obtain a target open source project whose first similarity with the project to be detected is greater than a first set threshold, and then further determine the homology relationship between the project to be detected and the target open source project according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at a preset detection level. According to the above homology detection process, the modified code files can also be accurately matched with homology open source projects, thereby improving the accuracy of code homology detection.

[0027] The defects existing in the above schemes and the proposed solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] To facilitate understanding of this embodiment, the SCA-based code homology detection method disclosed in the embodiment of the present disclosure is first introduced in detail. The executor of the SCA-based code homology detection method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities.

[0030] The SCA-based code homology detection method provided in the embodiment of the present disclosure is a method for performing homology detection on codes in a project to be detected based on a Software Component Analysis (SCA) method.

[0031] See also Figure 2 As shown, it is a flowchart of the code homology detection method based on SCA provided by an embodiment of the present disclosure, and the method includes S201 to S205, wherein: S201: Obtain directory fingerprint information of the item to be detected.

[0032] In the disclosed embodiment, a file directory tree of the project to be detected can be obtained. According to the file directory tree, each code file under the directory can be determined. Then, text hash calculations can be performed on each code file to obtain a hash value of each code file. After obtaining the hash value of each code file, the hash value of each code file can be sorted. Then, the hash values ​​of the sorted code files can be processed by character splicing to obtain the hash value of the file directory tree, that is, the directory fingerprint information of the project to be detected.

[0033] S202: Determine whether there is an association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in a pre-selected and maintained code fingerprint library.

[0034] In the embodiment of the present disclosure, the directory fingerprint information of the project to be detected can be compared with the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library. The generation process of the code fingerprint library will be described in detail below.

[0035] Here, each open source project in the pre-selected and maintained code fingerprint library can be an open source project under each version. The code fingerprint library can store directory fingerprint information of open source projects of each version. It should be noted here that each open source project in the embodiment of the present disclosure refers to an open source project under each version.

[0036] The directory fingerprint information may include the root directory fingerprint information and the sub-directory fingerprint information of sub-directories at all levels.

[0037] The relationship between the project to be detected and any open source project (that is, any version of the open source project) may include the following situations: When the root directory fingerprint information of the project to be detected is the same as the root directory fingerprint information of any open source project, it can be determined that the project to be detected is completely identical to the any open source project.

[0038] When the root directory fingerprint information of the project to be detected is the same as any subdirectory fingerprint information of any open source project, it can be determined that the project to be detected is a subset of the any open source project, that is, the project to be detected is included in the any open source project.

[0039] When the fingerprint information of any subdirectory of the project to be detected is the same as the fingerprint information of the root directory of any open source project, it can be determined that the any open source project is a subset of the project to be detected, that is, the project to be detected includes the any open source project.

[0040] When the fingerprint information of any subdirectory of the project to be detected is the same as the fingerprint information of any subdirectory of any open source project, it can be determined that the project to be detected and the open source project have an intersection, that is, the project to be detected is partially the same as the open source project.

[0041] In one implementation, the project to be detected may not have the above association relationship with any open source project. In this case, step S203 may continue to be performed.

[0042] S203: When there is no association relationship between any of the open source projects and the project to be detected, obtain content fingerprint information of each code file in the project to be detected.

[0043] In the embodiments of the present disclosure, a hash calculation may be performed on the text content of each code file in the project to be detected to obtain a hash value of the text content, that is, content fingerprint information.

[0044] S204: According to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, a target open source project whose first similarity with the project to be detected is greater than a first set threshold is selected from each open source project.

[0045] According to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in the open source project, a target open source project having a first similarity with the project to be detected that is greater than a first set threshold can be screened from various open source projects.

[0046] In one embodiment, according to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, screening a target open source project whose first similarity with the project to be detected is greater than a first set threshold from each open source project can include: determining a fingerprint vector of the project to be detected according to the content fingerprint information of each code file in the project to be detected; the dimension of the fingerprint vector is the number of code files; the value of each element in the fingerprint vector is the content fingerprint information of the corresponding code file; determining a preselected open source project whose distance from the project to be detected is less than a second set threshold according to the fingerprint vector of the project to be detected and the fingerprint vectors of each open source project; screening a target open source project whose first similarity with the project to be detected is greater than the first set threshold from the preselected open source projects according to the directory information of the project to be detected and the local sensitive hash values ​​of each code file contained in the project to be detected, as well as the directory information of each preselected open source project and the local sensitive hash values ​​of each code file contained in the preselected open source project.

[0047] In the above implementation, after obtaining the content fingerprint information of each code file in the project to be detected, a fingerprint vector of the project to be detected can be generated according to the content fingerprint information of each code file. Here, the dimension of the fingerprint vector can be determined according to the number of code files, and the value of each element in the fingerprint vector can be determined according to the content fingerprint information of the code files.

[0048] The pre-selected and maintained code fingerprint library may store content fingerprint information of each code file of each open source project. For each open source project, a fingerprint vector of each open source project may be generated according to the content fingerprint information of each code file in the above manner.

[0049] Then, the distances between the fingerprint vector of the project to be detected and the fingerprint vectors of each open source project can be determined respectively. Then, based on the distances between the fingerprint vector of the project to be detected and the fingerprint vectors of each open source project, pre-selected open source projects whose corresponding distances are less than a second set threshold are determined from each open source project.

[0050] Then, the directory information of the project to be detected can be compared with the directory information of each pre-selected open source project to obtain a first comparison result; and the local sensitive hash value of each code file included in the project to be detected can be compared with the local sensitive hash value of each code file included in each pre-selected open source project to obtain a second comparison result.

[0051] According to the first comparison result and the second comparison result, the second similarity between the project to be detected and each pre-selected open source project can be determined, so as to select a target open source project whose first similarity with the project to be detected is greater than a first set threshold from the pre-selected open source projects.

[0052] The above directory information may include the structure information, size information and hierarchical relationship of the directory tree, etc. The above local sensitive hash value is simhash.

[0053] The selected target open source projects can be used for homology comparison with the projects to be tested.

[0054] S205: Determine the homology relationship between the project to be detected and the target open source project based on the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level.

[0055] In one implementation, based on syntax fingerprint information of each code file of the target open source project at a preset detection level stored in the code fingerprint library and syntax tree fingerprint information of each code file of the project to be detected at a preset detection level, homology detection can be performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result; when the homology detection result indicates that the code file of the project to be detected and the code file of the target open source project are the same source code files, a second similarity between the project to be detected and the target open source project is determined based on content fingerprint information of each of the same source code files; and based on the second similarity, a homology relationship between the project to be detected and the target open source project is determined.

[0056] In the above implementation, the syntax fingerprint information of the code files in the target open source project stored in the code fingerprint library at a preset detection level can be compared with the syntax tree fingerprint information of the code files in the project to be detected at a preset detection level to obtain a homology detection result of whether the code files of the target open source project and the code files of the project to be detected are the same source code files.

[0057] After obtaining the homology detection result of whether the code file of the target open source project and the code file of the project to be detected are the same source code files, the second similarity between the project to be detected and the target open source project can be determined based on the content fingerprint information of each same source code file; then, based on the second similarity, the homology relationship between the project to be detected and the target open source project is determined.

[0058] In a specific implementation process, the component to which the project to be detected belongs, or the components included, can be determined according to the second similarity and the correspondence between the open source projects and components included in the code fingerprint library.

[0059] The above-mentioned preset detection level can be set according to the detection requirements of the user. In one embodiment, the preset detection level can include file-level detection, function-level detection or fragment-level detection.

[0060] The syntax tree fingerprint information under file-level detection is file-level syntax tree fingerprint information. File-level detection can be to compare the detected project with the target open source project based on the file-level syntax tree fingerprint information.

[0061] The syntax tree fingerprint information under function-level detection is the function-level syntax tree fingerprint information. Function-level detection can be to compare the source of the detected project with the target open source project based on the function-level syntax tree instruction information.

[0062] The syntax tree fingerprint information under the segment level detection is the segment level syntax tree fingerprint information. The segment level detection can be to compare the homology between the project to be detected and the target open source project according to the segment level syntax tree instruction information.

[0063] The detection accuracy of the above-mentioned fragment-level detection is greater than the detection accuracy of the function-level detection; and the detection accuracy of the function-level detection is greater than the detection accuracy of the file-level detection.

[0064] In one embodiment, when the preset detection level is file-level detection, based on the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at the preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result, which may include: when the file-level syntax tree fingerprint information of any code file of the target open source project stored in the code fingerprint library is the same as the file-level syntax tree fingerprint information of any code file of the project to be detected, determining that the any code file of the target open source project and the any code file of the project to be detected are the same source code files.

[0065] In the above implementation, the file-level syntax tree fingerprint information of each code file in the project to be detected can be compared with the file-level syntax tree fingerprint information of any code file of the target open source project. For each code file in the project to be detected, if the file-level syntax tree fingerprint information of the code file is the same as the file-level syntax tree fingerprint information of any code file of the target open source project, it can be determined that the code file is the same as the any code file of the target open source project, that is, the code file and the any code file of the target open source project are the same source code file.

[0066] When it is determined that any code file of the target open source project is different from any code file of the project to be detected, it can be determined that the target open source project and the project to be detected are of different sources.

[0067] In one implementation, when the preset detection level is function-level detection or segment-level detection, file-level detection may be performed first, and then function-level detection or segment-level detection may be performed. Specifically, according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result, including: according to the file-level syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library and the file-level syntax tree fingerprint information of each code file of the project to be detected, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a first homology detection result; when the first homology detection result indicates that each code file of the project to be detected and each code file of the target open source project are not of the same source, according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at the preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a second homology detection result.

[0068] In the above implementation, based on the file-level syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library and the file-level syntax tree fingerprint information of each code file of the project to be detected, the code files of the project to be detected and the code files of the target open source project are subjected to homology detection to obtain the first homology detection result, i.e., the process of performing file-level detection. The specific process can be referred to above and will not be repeated here.

[0069] When the first homology detection result indicates that the code files of the project to be detected are from different sources from the code files of the target open source project, that is, any code file in the project to be detected is different from any code file in the target open source project, function-level detection or fragment-level detection continues to be performed.

[0070] When the preset detection level is function-level detection, the function-level syntax tree fingerprint information of each code file in the project to be detected can be compared with the function-level syntax tree fingerprint information of any code file in the target open source project. For each code file in the project to be detected, if the function-level syntax tree fingerprint information of the code file is the same as the function-level syntax tree fingerprint information of any code file in the target open source project, it can be determined that the code file is the same as the any code file in the target open source project, that is, the code file and the any code file in the target open source project are the same source code file.

[0071] When the preset detection level is fragment-level detection, the fragment-level syntax tree fingerprint information of each code file in the project to be detected can be compared with the fragment-level syntax tree fingerprint information of any code file in the target open source project. For each code file in the project to be detected, if the fragment-level syntax tree fingerprint information of the code file is the same as the fragment-level syntax tree fingerprint information of any code file in the target open source project, it can be determined that the code file is the same as the any code file in the target open source project, that is, the code file and the any code file in the target open source project are the same source code file.

[0072] In one embodiment, when the preset detection level is function-level detection, based on the syntax tree fingerprint information of each code file of the target open source project at the preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a second homology detection result, including: if the number of valid lines of the longest common function-level syntax tree fingerprint information of any function in any code file of the project to be detected and any function in any code file of the target open source project is greater than the first preset valid line number, it is determined that any code file of the target open source project and any code file of the project to be detected are the same source code files.

[0073] In the above embodiment, for any code file in the project to be detected, if the number of valid lines of the longest common function-level syntax tree fingerprint information of any function of the code file and any function in any code file of the target open source project is greater than the first preset number of valid lines, it can be determined that the second homology detection result is that the code file and the any code file of the target open source project are the same source code file.

[0074] In one embodiment, when the preset detection level is fragment-level detection, based on the syntax tree fingerprint information of each code file of the target open source project at the preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a second homology detection result, including: if the number of valid lines of the longest common fragment-level syntax tree fingerprint information in any code file of the project to be detected and any code file of the target open source project is greater than the second preset valid line number, it is determined that any code file of the target open source project and any code file of the project to be detected are the same source code files.

[0075] In the above embodiment, for any code file of the project to be detected, if the number of valid lines of the longest common fragment-level syntax tree fingerprint information in the code file and any code file of the target open source project is greater than the second preset number of valid lines, it can be determined that the second homology detection result is that the code file and the any code file of the target open source project are the same source code file.

[0076] For example, the code file file1 in the project to be detected has a fragment-level syntax tree fingerprint information of [1,2,3,4,5], and the code file file2 in the target open source project has a fragment-level syntax tree fingerprint information of [2,3,4,5,6,7]. It can be determined that the four lines of fragment-level syntax tree fingerprint information common to code file file1 and code file file2 are [2,3,4,5], which are greater than the preset number of valid lines of 3, and it can be determined that code file file1 and code file file2 are the same source code files.

[0077] The following is an introduction to the generation process of the code fingerprint library.

[0078] In one implementation, the code fingerprint library may be determined according to the following steps: Step 1: Obtain version information, directory information, and shared project information for each version of the open source project.

[0079] Here, you can get various open source projects from the project database, or you can crawl various open source projects from the Internet.

[0080] Each open source project may include at least one version. Here, for each open source project, version information, directory information, and common project information of each version of the open source project may be obtained.

[0081] Among them, version information may include each submission record information commit of the version control system; directory information may include file directory tree, number of code files, etc.; shared project information may refer to the project information shared by each version of open source projects under each open source project, which may specifically include the uniform resource locator home_url for obtaining the homepage of the current site, the uniform resource locator doc_url for pointing to file resources, the uniform resource locator source_url for identifying the resource address, the uniform resource locator repo_url indicating the remote version library address, the homepage homepage, the number of times the GitHub repository has been forked by other users or organizations forks_count, description, authorization license file license and other information.

[0082] Step 2: For each version of the open source project, determine the target component to which the open source project belongs based on the version information, common project information, and component information of each component stored in the open source project. According to the target component to which each version of the open source project belongs, determine the correspondence between the open source project and the component.

[0083] Here, the database may store component information of each component, which may include component address information, component domain information Group Id, component name information Artifact Id, component version information Version, and the like.

[0084] Here, the open source project of this version can be matched with each component according to the version information of the open source project of this version, the common project information and the component information of each component stored, so as to obtain the target component to which the open source project of this version belongs. According to the target component to which each version of the open source project belongs, the corresponding relationship between the open source project and the component is determined.

[0085] Step 3: For each version of the open source project, generate directory fingerprint information of the version of the open source project based on the directory information of the version of the open source project and the content fingerprint information of each code file contained in the version of the open source project.

[0086] Here, for each version of the open source project, the content fingerprint information of each code file included in the version of the open source project can be sorted according to the file directory tree to obtain the arrangement order of the content fingerprint information of each code file, and then the content fingerprint information of each code file is spliced ​​according to the arrangement order of the content fingerprint information of each code file to obtain the spliced ​​content fingerprint information. Finally, the spliced ​​content fingerprint information is hashed to obtain the directory fingerprint information of the version of the open source project.

[0087] Step 4: Extract syntax tree structure information of each code file included in the open source project to obtain syntax tree structure information of each code file.

[0088] Here, you can extract the Abstract Syntax Tree (AST) of each code file contained in the open source project to obtain the syntax tree structure information of each code file.

[0089] Step 5. For each code file, generate file-level syntax tree fingerprint information of the code file according to the syntax tree structure information of the code file; determine each function in the code file, and determine the function-level syntax tree fingerprint information of each function according to the syntax tree structure information of the code file; determine the node of each line in the code file, and determine the syntax tree fingerprint information of each node according to the syntax tree structure information of the code file, and determine the fragment-level syntax tree fingerprint information according to the syntax tree fingerprint information of each node.

[0090] Here, for each code file, the syntax tree structure information of the code file can be enhanced, for example, parameter type comparison can be added, string functions can be equalized, empty statements such as empty_statement, single-line comments line_comment, multi-line comments block_comment and other nodes that do not affect the code logic can be removed, cross-line nodes and single-line nodes can be compared, characteristic characters in the string expression s-expression can be encoded in a preset form, etc., to obtain the file-level syntax tree fingerprint information of the code file.

[0091] For each code file, the syntax tree structure information of each function in the code file is obtained according to the syntax tree structure information of the code file, and the syntax tree structure information of each function is enhanced to obtain function-level syntax tree fingerprint information of each function.

[0092] For each code file, after obtaining the file-level syntax tree structure information, hash calculation can be performed on the nodes of each line in the code file to obtain the syntax tree fingerprint information of each node, and the fragment-level syntax tree fingerprint information can be determined based on the syntax tree fingerprint information of each node.

[0093] In the abstract syntax tree structure, there are many nodes. Each node is a code range. For example, [{2,1},{3,2}] means that the range of the node is from the second row and the first column to the third row and the second column. Valid nodes are all nodes except the comment information comment and the empty operation statement empty_statement. The line number of the starting position of each node is considered to be a valid line number. For example, the above [{2,1},{3,2}] means that the second line is a valid line number. Therefore, only the starting position of the node is considered valid here, and the end position does not participate in the calculation.

[0094] For function-level syntax tree fingerprint information, for example, the range of a function-level syntax tree fingerprint information is [{1,1},{10,8}]. Assuming that the valid nodes are only [{3,1},{4,2}] and [{6,3},{7,4}], then the valid line numbers are 3 and 6, a total of 2 lines, so the number of valid lines of the function-level syntax tree fingerprint information is 2.

[0095] For the fragment-level syntax tree fingerprint information, in the above example, the valid line numbers are 3 and 6, then the hash value obtained from line 3 and the hash value obtained from line 6 constitute the fragment-level syntax tree fingerprint information. For another example, the valid node node1 is between lines 1 and 5, the valid node node2 is between lines 2 and 4, the valid node node3 is between lines 2 and 5, and the valid node node4 is between lines 3 and 4. Then the first line calculates the hash value of the valid node node1 to be h1. The second line calculates the hash value of the concatenation of the valid node node2 and the valid node node3 to be h2, and the third line calculates the hash value of the valid node node4 to be h3. There is no starting position of the valid node node from the fourth to the fifth lines, and they are considered invalid lines. Then you can get the continuous hash list [h1,h2,h3] corresponding to the code file, which is the fragment-level syntax tree fingerprint information.

[0096] Step 6: Store the correspondence between open source projects and components, the syntax tree structure information of each code file, the file-level syntax tree fingerprint information of each code file, the function-level syntax tree fingerprint information of each function in each code file, and the fragment-level syntax tree fingerprint information of each code file in the code fingerprint library.

[0097] Here, in the specific storage process, the content fingerprint information of each code file can also be generated, and a set of code files with the same content fingerprint information can be determined; the file-level syntax tree fingerprint information, function-level syntax tree fingerprint information, and fragment-level syntax tree fingerprint information of any code file in the code file set are stored in the code fingerprint library. When the content fingerprint information of different code files is the same, these code files can be considered to be the same, so only the above information of any code file can be stored in the code fingerprint library.

[0098] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0099] Based on the same inventive concept, an SCA-based code homology detection device corresponding to the SCA-based code homology detection method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned SCA-based code homology detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0100] Reference Figure 3 FIG. 1 is a schematic diagram of a code homology detection device based on SCA provided in an embodiment of the present disclosure, wherein the device includes: The first acquisition module 301 is used to acquire directory fingerprint information of the item to be detected; A first determination module 302 is used to determine whether there is an association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library; The second acquisition module 303 is used to acquire content fingerprint information of each code file in the project to be detected when there is no association relationship between any of the open source projects and the project to be detected; A screening module 304 is used to screen a target open source project whose first similarity with the project to be detected is greater than a first set threshold from each open source project according to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project; The second determination module 305 is used to determine the homology relationship between the project to be detected and the target open source project based on the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level.

[0101] In an optional implementation manner, the device further includes a third determining module, configured to: Obtain version information, directory information, and shared project information for each version of open source projects; For each version of the open source project, determine the target component to which the open source project of this version belongs based on the version information of the open source project of this version, the common project information, and the component information of each component stored; determine the correspondence between the open source project and the component based on the target component to which each version of the open source project belongs; For each version of the open source project, generate directory fingerprint information of the open source project of this version according to the directory information of the open source project of this version and the content fingerprint information of each code file contained in the open source project of this version; Extract syntax tree structure information from each code file contained in the open source project to obtain syntax tree structure information of each code file; For each code file, generate file-level syntax tree fingerprint information of the code file according to the syntax tree structure information of the code file; determine each function in the code file, and determine the function-level syntax tree fingerprint information of each function according to the syntax tree structure information of the code file; determine the node of each line in the code file, and determine the syntax tree fingerprint information of each node according to the syntax tree structure information of the code file, and determine the fragment-level syntax tree fingerprint information according to the syntax tree fingerprint information of each node; The correspondence between open source projects and components, the syntax tree structure information of each code file, the file-level syntax tree fingerprint information of each code file, the function-level syntax tree fingerprint information of each function in each code file, and the fragment-level syntax tree fingerprint information of each code file are stored in the code fingerprint library.

[0102] In an optional implementation manner, the third determining module is specifically configured to: Generate content fingerprint information of each code file, and determine a set of code files with the same content fingerprint information; The file-level syntax tree fingerprint information, function-level syntax tree fingerprint information and fragment-level syntax tree fingerprint information of any code file in the code file set are stored in a code fingerprint library.

[0103] In an optional implementation, based on the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, a target open source project having a first similarity with the project to be detected greater than a first set threshold is selected from each open source project, including: Determine a fingerprint vector of the project to be detected according to the content fingerprint information of each code file in the project to be detected; the dimension of the fingerprint vector is the number of code files; the value of each element in the fingerprint vector is the content fingerprint information of the corresponding code file; Determine, according to the fingerprint vector of the project to be detected and the fingerprint vectors of each open source project, a preselected open source project whose distance from the project to be detected is less than a second set threshold; According to the directory information of the project to be detected and the local sensitive hash values ​​of each code file contained in the project to be detected, as well as the directory information of each pre-selected open source project and the local sensitive hash values ​​of each code file contained in each pre-selected open source project, a target open source project whose first similarity with the project to be detected is greater than a first set threshold is screened from the pre-selected open source projects.

[0104] In an optional implementation manner, the second determining module 305 is specifically configured to: According to the syntax tree fingerprint information of each code file of the target open source project at a preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result; When the homology detection result indicates that the code file of the project to be detected and the code file of the target open source project are the same source code files, determining a second similarity between the project to be detected and the target open source project according to content fingerprint information of each of the same source code files; According to the second similarity, a homology relationship between the project to be detected and the target open source project is determined.

[0105] In an optional implementation, the preset detection level includes file-level detection, function-level detection or fragment-level detection; the detection accuracy of the fragment-level detection is greater than the detection accuracy of the function-level detection; the detection accuracy of the function-level detection is greater than the detection accuracy of the file-level detection; The syntax tree fingerprint information under the file level detection is file level syntax tree fingerprint information; The syntax tree fingerprint information under the function level detection is function level syntax tree fingerprint information; The syntax tree fingerprint information under the segment level detection is segment level syntax tree fingerprint information.

[0106] In an optional implementation, the preset detection level is file-level detection; The second determining module 305 is specifically configured to: When the file-level syntax tree fingerprint information of any code file of the target open source project stored in the code fingerprint library is the same as the file-level syntax tree fingerprint information of any code file of the project to be detected, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code file.

[0107] In an optional implementation manner, the preset detection level is function-level detection or fragment-level detection; The second determination module 305 is specifically used to: perform homology detection on each code file of the project to be detected and each code file of the target open source project according to the file-level syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library and the file-level syntax tree fingerprint information of each code file of the project to be detected, to obtain a first homology detection result; When the first homology detection result indicates that the code files of the project to be detected are from different sources from the code files of the target open source project, a homology detection is performed on the code files of the project to be detected and the code files of the target open source project according to the syntax tree fingerprint information of the code files of the target open source project at a preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of the code files of the project to be detected at the preset detection level to obtain a second homology detection result.

[0108] In an optional implementation manner, when the preset detection level is function level detection, the second determination module 305 is specifically configured to: If the number of valid lines of the longest common function-level syntax tree fingerprint information of any function in any code file of the project to be detected and any function in any code file of the target open source project is greater than the first preset number of valid lines, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code files.

[0109] In an optional implementation manner, when the preset detection level is segment level detection, the second determination module 305 is specifically configured to: If the number of valid lines of the longest common fragment-level syntax tree fingerprint information in any code file of the project to be detected and any code file of the target open source project is greater than the second preset number of valid lines, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code files.

[0110] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0111] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 4 As shown, it is a schematic diagram of the structure of a computer device 400 provided in an embodiment of the present disclosure, including a processor 401, a memory 402, and a bus 403. Among them, the memory 402 is used to store execution instructions, including a memory 4021 and an external memory 4022; the memory 4021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 401, and the data exchanged with the external memory 4022 such as a hard disk. The processor 401 exchanges data with the external memory 4022 through the memory 4021. When the computer device 400 is running, the processor 401 communicates with the memory 402 through the bus 403, so that the processor 401 executes the following instructions: Get the directory fingerprint information of the project to be detected; Determine whether there is an association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library; In the case that there is no association relationship between any of the open source projects and the project to be detected, obtaining content fingerprint information of each code file in the project to be detected; According to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, a target open source project whose first similarity with the project to be detected is greater than a first set threshold is selected from each open source project; According to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, the homology relationship between the project to be detected and the target open source project is determined.

[0112] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the SCA-based code homology detection method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0113] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the SCA-based code homology detection method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0114] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0115] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0119] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. The code homology detection method based on SCA is characterized by: include: Get the directory fingerprint information of the project to be detected; Determine whether there is an association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library; In the case that there is no association relationship between any of the open source projects and the project to be detected, obtaining content fingerprint information of each code file in the project to be detected; According to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, a target open source project whose first similarity with the project to be detected is greater than a first set threshold is selected from each open source project; According to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level, and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, the homology relationship between the project to be detected and the target open source project is determined.

2. The method according to claim 1, characterized in that The code fingerprint library is determined according to the following steps: Obtain version information, directory information, and shared project information for each version of open source projects; For each version of the open source project, determine the target component to which the open source project of this version belongs based on the version information of the open source project of this version, the common project information, and the component information of each component stored; determine the correspondence between the open source project and the component based on the target component to which each version of the open source project belongs; For each version of the open source project, generate directory fingerprint information of the open source project of this version according to the directory information of the open source project of this version and the content fingerprint information of each code file contained in the open source project of this version; Extract syntax tree structure information from each code file contained in the open source project to obtain syntax tree structure information of each code file; For each code file, generate file-level syntax tree fingerprint information of the code file according to the syntax tree structure information of the code file; Determine each function in the code file, and determine function-level syntax tree fingerprint information of each function according to the syntax tree structure information of the code file; Determine the node of each line in the code file, and determine the syntax tree fingerprint information of each node according to the syntax tree structure information of the code file, and determine the fragment-level syntax tree fingerprint information according to the syntax tree fingerprint information of each node; The correspondence between open source projects and components, the syntax tree structure information of each code file, the file-level syntax tree fingerprint information of each code file, the function-level syntax tree fingerprint information of each function in each code file, and the fragment-level syntax tree fingerprint information of each code file are stored in the code fingerprint library.

3. The method according to claim 2, characterized in that The file-level syntax tree fingerprint information of each code file, the function-level syntax tree fingerprint information of each function in each code file, and the fragment-level syntax tree fingerprint information of each code file are stored in the code fingerprint library, including: Generate content fingerprint information of each code file, and determine a set of code files with the same content fingerprint information; The file-level syntax tree fingerprint information, function-level syntax tree fingerprint information and fragment-level syntax tree fingerprint information of any code file in the code file set are stored in a code fingerprint library.

4. The method according to claim 1, characterized in that: According to the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project, a target open source project having a first similarity with the project to be detected greater than a first set threshold is selected from each open source project, including: Determine a fingerprint vector of the project to be detected according to the content fingerprint information of each code file in the project to be detected; the dimension of the fingerprint vector is the number of code files; the value of each element in the fingerprint vector is the content fingerprint information of the corresponding code file; Determine, according to the fingerprint vector of the project to be detected and the fingerprint vectors of each open source project, a preselected open source project whose distance from the project to be detected is less than a second set threshold; According to the directory information of the project to be detected and the local sensitive hash values ​​of each code file contained in the project to be detected, as well as the directory information of each pre-selected open source project and the local sensitive hash values ​​of each code file contained in each pre-selected open source project, a target open source project whose first similarity with the project to be detected is greater than a first set threshold is screened from the pre-selected open source projects.

5. The method according to claim 1, characterized in that Determining the homology relationship between the project to be detected and the target open source project according to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, including: According to the syntax tree fingerprint information of each code file of the target open source project at a preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result; When the homology detection result indicates that the code file of the project to be detected and the code file of the target open source project are the same source code files, determining a second similarity between the project to be detected and the target open source project according to content fingerprint information of each of the same source code files; According to the second similarity, a homology relationship between the project to be detected and the target open source project is determined.

6. The method according to claim 5, characterized in that The preset detection level includes file-level detection, function-level detection or fragment-level detection; the detection accuracy of the fragment-level detection is greater than the detection accuracy of the function-level detection; the detection accuracy of the function-level detection is greater than the detection accuracy of the file-level detection; The syntax tree fingerprint information under the file level detection is file level syntax tree fingerprint information; The syntax tree fingerprint information under the function level detection is function level syntax tree fingerprint information; The syntax tree fingerprint information under the segment level detection is segment level syntax tree fingerprint information.

7. The method according to claim 6, characterized in that The preset detection level is file-level detection; According to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result, including: When the file-level syntax tree fingerprint information of any code file of the target open source project stored in the code fingerprint library is the same as the file-level syntax tree fingerprint information of any code file of the project to be detected, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code file.

8. The method according to claim 6, characterized in that The preset detection level is function level detection or fragment level detection; According to the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a homology detection result, including: According to the file-level syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library and the file-level syntax tree fingerprint information of each code file of the project to be detected, a homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a first homology detection result; When the first homology detection result indicates that the code files of the project to be detected are from different sources from the code files of the target open source project, a homology detection is performed on the code files of the project to be detected and the code files of the target open source project according to the syntax tree fingerprint information of the code files of the target open source project at a preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of the code files of the project to be detected at the preset detection level to obtain a second homology detection result.

9. The method according to claim 8, characterized in that In the case where the preset detection level is function-level detection, according to the syntax tree fingerprint information of each code file of the target open source project at the preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a second homology detection result, including: If the number of valid lines of the longest common function-level syntax tree fingerprint information of any function in any code file of the project to be detected and any function in any code file of the target open source project is greater than the first preset number of valid lines, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code files.

10. The method according to claim 8, characterized in that In the case where the preset detection level is fragment-level detection, according to the syntax tree fingerprint information of each code file of the target open source project at the preset detection level stored in the code fingerprint library and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level, homology detection is performed on each code file of the project to be detected and each code file of the target open source project to obtain a second homology detection result, including: If the number of valid lines of the longest common fragment-level syntax tree fingerprint information in any code file of the project to be detected and any code file of the target open source project is greater than the second preset number of valid lines, it is determined that the any code file of the target open source project and the any code file of the project to be detected are the same source code files.

11. The code homology detection device based on SCA is characterized by: include: The first acquisition module is used to obtain directory fingerprint information of the item to be detected; A first determination module is used to determine whether there is an association relationship between the project to be detected and each open source project according to the directory fingerprint information of the project to be detected and the directory fingerprint information of each open source project stored in the pre-selected and maintained code fingerprint library; A second acquisition module is used to acquire content fingerprint information of each code file in the project to be detected when there is no association relationship between any of the open source projects and the project to be detected; a screening module, configured to screen, from each open source project, a target open source project whose first similarity with the project to be detected is greater than a first set threshold value, based on the directory information of the project to be detected and the content fingerprint information of each code file in the project to be detected, as well as the directory information of each open source project and the content fingerprint information of each code file in each open source project; The second determination module is used to determine the homology relationship between the project to be detected and the target open source project based on the syntax tree fingerprint information of each code file of the target open source project stored in the code fingerprint library at a preset detection level and the syntax tree fingerprint information of each code file of the project to be detected at the preset detection level.

12. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the SCA-based code homology detection method as described in any one of claims 1 to 10 are performed.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the SCA-based code homology detection method according to any one of claims 1 to 10 are executed.

Images