Code detection and analysis method and computing equipment

Abstract

The invention provides a code detection and analysis method, and proposes that each second code snippet included in a public code database belongs to at least two different open source code files, and each open source code file belongs to code snippets in at least two open source code warehouses. If the code snippets in the common code database include the code snippets in the user code file, it is determined that the code snippets of the user are common code snippets. The codes in the common code database collect the common code snippets from different open source code files and different open source code warehouses, so that when a user detects whether the code snippets in the open source code files belong to the open source codes of a certain protocol or not, the false alarm rate can be reduced; and the code detection accuracy and the code development efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of code detection, and more specifically, to a method and computing device for code detection analysis. Background Art

[0002] Open source code, also known as source code disclosure, refers to a software release model, usually referred to as "open source code". Usually only the author of the software or the copyright owner owns the source code of the program. Some software authors will make the source code public, which is called "source code disclosure", but this does not necessarily meet the definition and conditions of "open source" because the author may set restrictions on the conditions for disclosing the source code. For example, if a user uses the code of the GNU General Public License (GPL) agreement in his own code, the code developed by the user himself also needs to be "source code disclosed". If the user does not want to open source, he cannot use the code of the GPL agreement. Therefore, users need to perform reference detection and analysis on the code they develop to check whether the code they develop references the code in the GPL agreement.

[0003] Code clone detection is a technology used to identify similar parts in source code. It plays an important role in the field of software engineering. It can help developers identify repeated, redundant or maintenance-problem-causing code snippets. If it is found that several lines of the code they developed are similar to open source code, it is considered that there is a snippet reference. This method can find suspected open source snippet references relatively efficiently, but there may also be some misjudgments. For example, under normal circumstances, if the comment at the beginning of the code is license information, since the text of the license is basically the same, this method will also determine the license information as a snippet reference; for example, for some common shorter functions such as "get", "set", etc., there will be multiple lines of the same code, and this method will also determine it as a snippet reference. It can be seen that the accuracy of existing open source code detection schemes is not high.

[0004] Therefore, a method for code detection and analysis is needed to improve the accuracy of code detection. Summary of the invention

[0005] The present application provides a method for code detection and analysis, which can improve the accuracy of code detection and the efficiency of code development.

[0006] In a first aspect, a method for code detection and analysis is provided. In a possible implementation, the method can be applied to a cloud service system. The method can be executed by a cloud management platform (or a communication device in a cloud management platform), or by a component (such as a chip or circuit) of the cloud management platform (or a communication device in a cloud management platform). In another possible implementation, the method can be executed by a computing device (such as a computer), or by a component (such as a chip or circuit) of a computing device.

[0007] The method includes: obtaining a first code snippet, the first code snippet is a code snippet in a user code file that is similar to an open source code included in an open source code repository, the number of code lines included in the first code snippet is N, and N is an integer greater than or equal to 2; if the first code snippet is included in a public code database, determining that the first code snippet in the user code file is a public code snippet, wherein the public code database includes multiple second code snippets, each of the multiple second code snippets belongs to at least two different open source code files, each of the at least two different open source code files belongs to at least two different open source code repositories, and the number of code lines included in each second code snippet is N.

[0008] In this application, a "code snippet" can be understood as multiple consecutive lines of code.

[0009] In the present application, the word "similar" in "a code snippet in a user code file that is similar to an open source code included in an open source code repository" can be understood as, for example, multiple consecutive lines of a code snippet in a user code file are similar or identical to multiple consecutive lines in a certain open source file included in the open source code repository.

[0010] The "second code snippet" in this application may come from the open source code. Alternatively, it can be understood that any code snippet in the open source code may become the "second code snippet" in this application, as long as a code snippet in the open source code belongs to at least two different open source code files, and each of the at least two open source code files belongs to at least two different open source code repositories, it can become the "second code snippet".

[0011] It can also be understood that in this application, multiple different open source code files and multiple different open source code repositories all include the second code snippet. For example, code snippet #1 belongs to both open source code file #1, open source code file #2, and open source code file #3, and open source code file #1 belongs to both open source code repositories #A, open source code repositories #B, and open source code repositories #C; open source code file #2 belongs to both open source code repositories #A and open source code repositories #D; and open source code file #3 belongs to both open source code repositories #B and open source code repositories #C.

[0012] In this application, in a possible implementation, each open source code file corresponds to at least two different file names. For example, the file name of the open source code file #1 in the open source code warehouse #A is file_name#1, the file name of the open source code file #1 in the open source code warehouse #B is file_name#2, and the file name of the open source code file #1 in the open source code warehouse #C is file_name#3.

[0013] In this application, in a possible implementation, each open source code file corresponds to at least two different types of file licenses. For example, the file license type of the open source code file #1 in the open source code repository #A is license #1, the file name of the open source code file #1 in the open source code repository #B is license #2, and the file name of the open source code file #1 in the open source code repository #C is license #3.

[0014] Based on the above technical solution, the present application proposes that each second code snippet included in the public code database belongs to at least two different open source code files, and each open source code file belongs to a code snippet in at least two open source code repositories. If the code snippet in the public code database includes a code snippet in a user code file, the user's code snippet is determined to be a public code snippet. Since the code in the public code database in the present application collects public code snippets from different files and different repositories, when the user detects whether the code snippet in his or her open source code file belongs to the open source code of a certain protocol, the false alarm rate can be reduced, and the accuracy of code detection and the efficiency of code development can be achieved.

[0015] In a possible implementation, each open source code file corresponds to at least two different file names, and / or each open source code file corresponds to at least two different types of file licenses.

[0016] Based on the above technical solution, in this application, in different open source code repositories, the same open source code files may have different file names and / or different types of file licenses. These features further illustrate that the code snippet is public code.

[0017] In a possible implementation, the method further includes: determining, based on the user code file and the open source code repository, a code snippet in the user code file that is similar to the open source code included in the open source code repository.

[0018] In a possible implementation, the method further includes: determining a second code snippet in an open source code repository; and generating a public code database based on multiple second code snippets.

[0019] Based on the above technical solution, in this application, common code snippets can be first mined in the open source code to generate a common code database, which is convenient for subsequent comparison of whether the code snippets in the user code files are common codes.

[0020] In one possible implementation, each open source code file includes at least one third code snippet, and the third code snippet includes N lines of code. Determining the second code snippet includes: determining a file hash value corresponding to each open source code file according to at least one third code snippet included in each code file; determining a first mapping relationship according to each third code snippet in each open source code file and the file hash value corresponding to each open source code file, wherein the first mapping relationship is a mapping relationship between the third code snippet and the file hash value, wherein the file hash value is the file hash value corresponding to the open source code file to which the third code snippet belongs; determining a fragment hash value corresponding to each third code snippet according to the N lines of code included in each third code snippet; determining a fragment hash value corresponding to each third code snippet according to the first mapping relationship and the fragment hash value corresponding to each third code snippet. Determine a second mapping relationship, where the second mapping relationship is a mapping relationship between the fragment hash value corresponding to each third code fragment and the file hash value; determine, based on the second mapping relationship, the number of different file hash values ​​corresponding to the same fragment hash value as a first number; determine a third mapping relationship, where the third mapping relationship is a mapping relationship between open source code files and their corresponding open source code repositories; determine, based on the third mapping relationship, the number of different open source code repositories to which the open source code files corresponding to each file hash value in the different file hash values ​​in the first number belong as a second number; when the first number is greater than or equal to a first threshold, and / or the second number is greater than or equal to a second threshold, determine the third code fragment corresponding to the same fragment hash value as a second code fragment, wherein the first threshold and the second threshold are both integers greater than or equal to 2.

[0021] Based on the above technical solution, in this application, by calculating the file hash value corresponding to the open source code file and the fragment hash value corresponding to the code fragment, it can be determined that the same code fragment appears in multiple open source code files, and based on the mapping relationship between the open source code file and the open source code repository to which it belongs, it is determined that the open source code file appears in multiple open source code repositories, so that it can be quickly determined whether a code fragment in the open source code is public code.

[0022] In a possible implementation, the method further includes: determining a fourth mapping relationship according to the file hash value and the first information, the fourth mapping relationship being a mapping relationship between the file hash value and the first information, wherein the first information includes the file name of the open source code file and / or the type of the file license of the open source code file; determining, according to the fourth mapping relationship, the number of different file names of the open source code file corresponding to each file hash value in the different file hash values ​​in the first quantity as a third number, and / or, according to the fourth mapping relationship, determining the number of different types of file licenses of the open source code file corresponding to each file hash value in the different file hash values ​​in the first quantity as a fourth number; when the first number is greater than or equal to a first threshold, and / or when the second number is greater than or equal to a second threshold, determining the third code fragment corresponding to the same fragment hash value as the second code fragment, including: when the first number is greater than or equal to the first threshold, and the second number is greater than or equal to the second threshold, and the third number is greater than or equal to the third threshold, and / or when the fourth number is greater than or equal to the fourth threshold, determining the third code fragment corresponding to the same fragment hash value as the second code fragment, wherein the third threshold and the fourth threshold are both integers greater than or equal to 2.

[0023] Based on the above technical solution, the present application can further determine the number of file names and the number of file license types of open source code files in different open source code repositories, thereby making the final screened public code more accurate.

[0024] In a possible implementation, the method also includes: determining a fragment hash value corresponding to the first code fragment based on N lines of code included in the first code fragment; if the first code fragment is included in a public code database, determining that the first code fragment in the user code file is a public code fragment, including: if the fragment hash values ​​corresponding to each of multiple second code fragments included in the public code database include the fragment hash value corresponding to the first code fragment, determining that the first code fragment is a public code fragment.

[0025] In a second aspect, the present application proposes a computing device, which is used to execute the method of the first aspect above. Specifically, the device may include a unit and / or module for executing a code detection and analysis method proposed in the present application, such as a transceiver unit and / or a processing unit. For example, in the present application, the computing device can be applied to a cloud management platform. Or, for example, the computing device is a cloud management platform.

[0026] In a third aspect, the present application provides a computing device, the device comprising: at least one processor, configured to execute a computer program or instruction stored in a memory to perform the method of the first aspect. Optionally, the device further comprises a memory, configured to store a computer program or instruction. Optionally, the device further comprises a communication interface, and the processor reads the computer program or instruction stored in the memory through the communication interface.

[0027] In a possible implementation manner, the computing device is a computing device used to implement the functions of the method of the first aspect in a chip.

[0028] In another possible implementation, the computing device is a chip, a chip system or a circuit for implementing the solution of the above-mentioned first aspect in a chip.

[0029] In one implementation, the device is a communication device used in a cloud management platform for implementing the functions of the method of the first aspect in a chip. Alternatively, the device is a cloud management platform.

[0030] In another implementation, the device is a chip, a chip system or a circuit used in a cloud management platform for implementing the functions of the first aspect in a chip.

[0031] In a fourth aspect, the present application provides a processor, comprising: an input circuit, an output circuit and a processing circuit. The processing circuit is used to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method of the first aspect.

[0032] In the specific implementation process, the processor can be one or more chips, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit can be received and input by, for example, but not limited to, a transceiver, and the signal output by the output circuit can be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit can be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation methods of the processor and various circuits.

[0033] For the operations such as sending and acquiring / receiving involved in the processor, unless otherwise specified, or unless they conflict with their actual function or internal logic in the relevant description, they can be understood as operations such as processor output, reception, input, etc., or as sending and receiving operations performed by the radio frequency circuit and antenna, and this application does not limit this.

[0034] In a fifth aspect, a processing device is provided, comprising a processor and a memory. The processor is used to read instructions stored in the memory, and can receive signals through a transceiver and transmit signals through a transmitter to execute the method of the first aspect.

[0035] Optionally, the number of the processors is one or more, and the number of the memories is one or more.

[0036] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0037] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips respectively. The embodiments of the present application do not limit the type of memory and the setting method of the memory and the processor.

[0038] It should be understood that the related data interaction process, such as sending indication information, can be a process of outputting indication information from the processor, and receiving capability information can be a process of receiving input capability information by the processor. Specifically, the data output by the processor can be output to the transmitter, and the input data received by the processor can come from the transceiver. Among them, the transmitter and the transceiver can be collectively referred to as a transceiver.

[0039] The processing device in the fifth aspect may be one or more chips. The processor in the processing device may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor or located outside the processor and exist independently.

[0040] In a sixth aspect, a computing cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in any possible implementation manner in the first aspect.

[0041] Optionally, the processor may be a general-purpose processor, which may be implemented by hardware or software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor or located outside the processor and exist independently.

[0042] In a seventh aspect, a computer-readable storage medium is provided, wherein the computer-readable medium stores a program code for execution by a device, wherein the program code includes a method for executing the above-mentioned first aspect.

[0043] According to an eighth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method according to the first aspect.

[0044] In a ninth aspect, a chip system is provided, comprising a processor for calling and running a computer program from a memory, so that a device equipped with the chip system executes the method of the first aspect.

[0045] In a tenth aspect, a cloud service system is provided, which includes a cloud management platform, wherein the cloud management platform is used to execute the method in any possible implementation of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of a system architecture applicable to the present application.

[0047] Figure 2 It is a schematic flow chart of a code detection and analysis method 200 provided in the present application.

[0048] Figure 3 It is a schematic diagram of another system architecture applicable to the present application.

[0049] Figure 4 It is a schematic diagram of a cloud service scenario used in this application.

[0050] Figure 5 is a schematic block diagram of a computing device 500 provided in the present application.

[0051] Figure 6 It is a schematic block diagram of the communication device 600 provided in this application.

[0052] Figure 7 This is a schematic diagram of the architecture of a computing device cluster provided by this application.

[0053] Figure 8It is a schematic diagram of the connection between computing devices 700A and 700B provided by the present application through a network. DETAILED DESCRIPTION

[0054] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0055] In order to facilitate understanding of the technical solution of the present application, some professional terms involved in the present application are first introduced below.

[0056] 1. Open source code

[0057] "Open source" is also called source code disclosure, which refers to a software release model, usually referred to as "open source code". General software can only obtain compiled binary executable files, and usually only the author of the software or the copyright owner owns the source code of the program. Some software authors will make the source code public, but this does not necessarily meet the definition and conditions of "open source" because the author may set conditional restrictions on the disclosure of the source code, such as limiting the objects that can read the source code, limiting derivatives, etc. For example, if a user uses the code of the GNU General Public License (GPL) agreement in his own code, the code developed by the user himself also needs to be "open source". If the user does not want to open source, he cannot use the code of the GPL agreement. Therefore, users need to conduct reference analysis on the code they develop to check whether the code they develop refers to the code in the GPL agreement.

[0058] 2. Hash values

[0059] "Hash value" refers to the value calculated using a hash function. A hash function is a method of creating a small digital "fingerprint" from any data. A hash function compresses a message or data into a summary, making the data smaller and fixing the format of the data. The corresponding "summary" is calculated based on the data content, and the data can be distinguished from other data based on this summary. Usually, the hash value of a file can be calculated to compare whether the content of the file is consistent.

[0060] Code clone detection is a technology used to identify similar parts in source code. It plays an important role in the field of software engineering. It can help developers identify code snippets that are repeated, redundant, or may cause maintenance problems. If it is found that several lines of self-developed code are similar to open source code, it is considered that there is a snippet reference. This method can find suspected open source snippet references relatively efficiently, but there may be some misjudgments. For example, if the comment at the beginning of the code is license information, since the text of the license is basically the same, the method will also judge the license information as a snippet reference; for example, for some common shorter functions such as get, set, etc., there will be multiple lines of the same code, and this method will also judge it as a snippet reference. At this time, the user needs to confirm one by one and delete the false positive results. It can be seen that the accuracy of existing open source code detection solutions is not high.

[0061] 3. Cloud management platform

[0062] The cloud management platform can be used to provide an access interface (for example, an interface or an application programming interface (API). The tenant can operate the client remote access access interface to register a cloud account and password on the cloud management platform and log in to the cloud management platform. After the cloud management platform successfully authenticates the cloud account and password, the tenant can further pay to select and purchase a virtual machine with specific specifications (processor, memory, disk) on the cloud management platform. After the payment is successful, the cloud management platform provides the remote login account and password of the purchased virtual machine, and the client can remotely log in to the virtual machine to install and run the tenant's application in the virtual machine.

[0063] In view of this, the present application provides a method for code detection and analysis, which proposes that each second code snippet included in a public code database belongs to at least two different open source code files, and each open source code file belongs to a code snippet in at least two open source code repositories. If the code snippet in the public code database includes a code snippet in a user code file, it is determined that the user's code snippet is a public code snippet. Since the code in the public code database in the present application collects public code snippets from different open source code files and different code repositories, when a user detects whether a code snippet in his or her own open source code file belongs to the open source code of a certain protocol, the false alarm rate can be reduced, and the accuracy of code detection and the efficiency of code development can be achieved.

[0064] Figure 1 This is a schematic diagram of a scenario applicable to this application, such as Figure 1As shown, the "Fragment Reference Analysis Engine" can compare the user's code files with the source code in the entire open source code repository based on the open source code library. The user sends the code file written by the user to the "Scanning Service Module" to start scanning. The "Scanning Service Module" calls the "Fragment Reference Analysis Engine" module. The "Fragment Reference Analysis Engine Module" scans the user's code to find code fragments in the user's code that are similar to the open source code. The "Scanning Service Module" writes the scanning results into the database, and the user further confirms and clarifies the scanning results through the "Confirmation Service Module".

[0065] Figure 2 is a schematic flow chart of a code detection and analysis method 200 proposed in this application, such as Figure 2 As shown, the dotted box represents that the steps are optional. In the present application, since the process of determining the second code fragment and constructing the public code database based on the second code fragment is not necessarily performed every time the method is executed, usually, 210 and 220 only need to be performed once, so 210 and 220 are optional steps, and the method includes:

[0066] Optionally, 210, a second code snippet is determined in an open source code repository.

[0067] In the present application, each second code snippet belongs to at least two different open source code files, each of the at least two open source code files belongs to at least two different open source code repositories, and each second code snippet includes N lines of code, where N is an integer greater than or equal to 2.

[0068] The "second code snippet" in this application may come from the open source code. Alternatively, it can be understood that any code snippet in the open source code may become the "second code snippet" in this application, as long as a code snippet in the open source code belongs to at least two different open source code files, and each of the at least two open source code files belongs to at least two different code repositories, it can become the "second code snippet".

[0069] It can also be understood that in this application, multiple different open source code files and multiple different open source code repositories all include the second code snippet. For example, code snippet #1 belongs to both open source code file #1, open source code file #2, and open source code file #3, and open source code file #1 belongs to both open source code repositories #A, open source code repositories #B, and open source code repositories #C; open source code file #2 belongs to both open source code repositories #A and open source code repositories #D; and open source code file #3 belongs to both open source code repositories #B and open source code repositories #C.

[0070] In this application, in a possible implementation, each open source code file corresponds to at least two different file names. For example, the file name of the open source code file #1 in the open source code warehouse #A is file_name#1, the file name of the open source code file #1 in the open source code warehouse #B is file_name#2, and the file name of the open source code file #1 in the open source code warehouse #C is file_name#3.

[0071] In this application, in a possible implementation, each open source code file corresponds to at least two different types of file licenses. For example, the file license type of the open source code file #1 in the open source code repository #A is license #1, the file name of the open source code file #1 in the open source code repository #B is license #2, and the file name of the open source code file #1 in the open source code repository #C is license #3.

[0072] The present application below introduces a method for determining a second code snippet in an open source code file. The method can relatively quickly determine the second code snippet in the open source code by calculating the open source code file hash value and the code snippet hash value.

[0073] In this application, "determining the second code snippet" can also be understood as a process of mining common code snippets in the open source code. For example, the process can be an offline computing process, for example, a distributed computing framework such as Spark can be used for computing, and the code can be read from the open source code to mine the code snippets to obtain a common code library. The specific steps are as follows:

[0074] Step 1: Traverse the open source code and preprocess the code in each open source code file in the open source code.

[0075] For example, the preprocessing operation may include one or more of the following: (1) dividing the code in the open source code file into uniform lines and removing blank lines; (2) identifying variable names through code analysis and replacing all variable names with a symbol, such as uniformly using "identifier" to represent variable names; (3) removing modifiers that are not related to the code logic, such as keywords such as "public" and "private".

[0076] Step 2: Segment the code included in each open source code file and determine the number of lines of code included in the code snippet.

[0077] For example, assuming that the smallest code snippet is 6 lines, the hash value of the code snippet can be calculated using 6 lines as a sliding window. For another example, assuming that the smallest code snippet is 10 lines, the hash value of the code snippet can be calculated using 10 lines as a sliding window.

[0078] It should be understood that in the present application, the number of lines of the code snippet can be flexibly determined by those skilled in the art based on actual conditions and is not limited.

[0079] Step 3: Determine the file hash value corresponding to each open source code file.

[0080] Assuming that each open source code file includes at least one third code snippet, a file hash value corresponding to each open source code file is calculated based on one or more third code snippets included in each file. That is, each open source code file includes multiple lines of code, and a hash value is obtained by calculating the multiple lines of code, and the hash value is the file hash value corresponding to the open source code file. It should be understood that the same open source code files have the same file hash value calculated. Specifically, the method for calculating the file hash value corresponding to each open source code file can refer to the existing method and will not be described in detail here.

[0081] For example, assuming that the open source code file #A includes three code snippets, namely: code snippet #A1, code snippet #A2, and code snippet #A3, the file hash value corresponding to the open source code file #A is determined by calculation to be file hash value #H1; assuming that the open source code file #B includes three code snippets, namely: code snippet #A1, code snippet #B1, and code snippet #B2, the file hash value corresponding to the open source code file #B is determined by calculation to be file hash value #H2; assuming that the open source code file #C includes three code snippets, namely: code snippet #A1, code snippet #A2, and code snippet #B2, the file hash value corresponding to the open source code file #C is determined by calculation to be file hash value #H3, as shown in Table 1 below.

[0082] Table 1

[0083] Open source code files File hashes Open Source Code File#A File hash #H1 Open Source Code File#B File hash #H2 Open Source Code File#C File hash #H3

[0084] Step 4: Determine the first mapping relationship.

[0085] In the present application, a first mapping relationship can be determined based on each third code snippet in each open source code file and the file hash value corresponding to each open source code file. The first mapping relationship is a mapping relationship between the third code snippet and the file hash value corresponding to the open source code file to which it belongs.

[0086] Taking the open source code file #A as an example, the first mapping relationship in the open source code file #A is "code snippet #A1—file hash value #H1", "code snippet #A2—file hash value #H1", and "code snippet #A3—file hash value #H1"; taking the open source code file #B as an example, the first mapping relationship in the open source code file #B is "code snippet #A1—file hash value #H2", "code snippet #B1—file hash value #H2", and "code snippet #B2—file hash value #H2"; taking the open source code file #C as an example, the first mapping relationship in the open source code file #C is "code snippet #A1—file hash value #H3", "code snippet #A2—file hash value #H3", and "code snippet #B2—file hash value #H3", as shown in Table 2 below.

[0087] Table 2

[0088]

[0089] Step 5: Determine the fragment hash value corresponding to each third code fragment.

[0090] Assume that the number of code lines included in each third code snippet is 6 (i.e., N is 6), then the snippet hash values ​​corresponding to each third code snippet can be calculated respectively. It should be understood that the same code snippets have the same snippet hash values. Specifically, the method for calculating the snippet hash value corresponding to each code snippet can refer to the existing method, which will not be described in detail here.

[0091] For example, assume that the fragment hash value corresponding to code fragment #A1 in open source code file #A is fragment hash value #A1, the fragment hash value corresponding to code fragment #A2 is fragment hash value #A2, and the fragment hash value corresponding to code fragment #A3 is fragment hash value #A3; assume that the fragment hash value corresponding to code fragment #A1 in open source code file #B is fragment hash value #A1, the fragment hash value corresponding to code fragment #B1 is fragment hash value #B1, and the fragment hash value corresponding to code fragment #B2 is fragment hash value #B2; assume that the fragment hash value corresponding to code fragment #A1 included in open source code file #C is fragment hash value #A1, the fragment hash value corresponding to code fragment #A2 is fragment hash value #A2, and the fragment hash value corresponding to code fragment #B2 is fragment hash value #B2, as shown in Table 3 below.

[0092] Table 3

[0093]

[0094] Step six: Determine the second mapping relationship.

[0095] The second mapping relationship is determined according to the first mapping relationship and the fragment hash value corresponding to each third code fragment, wherein the second mapping relationship is a mapping relationship between the fragment hash value corresponding to each third code fragment and the file hash value.

[0096] Taking the open source code file #A as an example, the second mapping relationship in the open source code file #A is "fragment hash value #A1—file hash value #H1", "fragment hash value #A2—file hash value #H1", and "fragment hash value #A3—file hash value #H1"; taking the open source code file #B as an example, the second mapping relationship in the open source code file #B is "fragment hash value #A1—file hash value #H2", "fragment hash value #B1—file hash value #H2", and "fragment hash value #B2—file hash value #H2"; taking the open source code file #C as an example, the second mapping relationship in the open source code file #C is "fragment hash value #A1—file hash value #H3", "fragment hash value #A2—file hash value #H3", and "fragment hash value #B2—file hash value #H3", as shown in Table 4 below.

[0097] Table 4

[0098] Fragment Hash Value File hashes Fragment hash value #A1 File hash #H1 Fragment hash value #A2 File hash #H1 Fragment hash value #A3 File hash #H1 Fragment hash value #A1 File hash #H2 Fragment hash value #B1 File hash #H2 Fragment hash value #B2 File hash #H2 Fragment hash value #A1 File hash #H3 Fragment hash value #A2 File hash #H3 Fragment hash value #B1 File hash #H3

[0099] Step 7: According to the second mapping relationship, determine that the number of different file hash values ​​corresponding to the same segment hash value is the first number.

[0100] For example, it can be seen from Table 4 that there are 3 file hash values ​​corresponding to fragment hash value #A1 (an example of the first quantity), namely file hash value #H1, file hash value #H2, and file hash value #H3; there are 2 file hash values ​​corresponding to fragment hash value #A2 (another example of the first quantity), namely file hash value #H1 and file hash value #H3; there are 2 file hash values ​​corresponding to fragment hash value #B1 (another example of the first quantity), namely file hash value #H2 and file hash value #H3.

[0101] Step 8: Determine the third mapping relationship.

[0102] In the present application, the third mapping relationship is a mapping relationship between an open source code file and an open source code repository to which the open source code file belongs.

[0103] Assume that open source code file #A belongs to open source code warehouse #1, open source code warehouse #2, open source code warehouse #3, and open source code warehouse #4; open source code file #B belongs to open source code warehouse #2, open source code warehouse #3, and open source code warehouse #4; open source code file #C belongs to open source code warehouse #1, open source code warehouse #3, and open source code warehouse #4, as shown in Table 5.

[0104] Table 5

[0105]

[0106] Step nine: According to the third mapping relationship, determine that the number of different open source code repositories to which the open source code files corresponding to each file hash value in the different file hash values ​​in the first number belong is the second number.

[0107] Exemplarily, there are three file hash values ​​corresponding to the fragment hash value #A1, namely file hash value #H1, file hash value #H2, and file hash value #H3. At this time, based on Table 1 and Table 5, it can be seen that the open source code file corresponding to the file hash value #H1 is the open source code file #A, and there are four open source code repositories to which the open source code file #A belongs (an example of the second number); the open source code file corresponding to the file hash value #H2 is the open source code file #B, and there are three open source code repositories to which the open source code file #B belongs (another example of the second number); the open source code file corresponding to the file hash value #H3 is the open source code file #C, and there are three open source code repositories to which the open source code file #C belongs (another example of the second number).

[0108] Step ten: when the first number is greater than or equal to the first threshold, and / or the second number is greater than or equal to the second threshold, determine the third code fragment corresponding to the same fragment hash value as the second code fragment.

[0109] In the present application, as long as either the first quantity or the second quantity meets the threshold condition, the third code snippet can be used as the second code snippet in the public code database.

[0110] In the present application, the first threshold and the second threshold are both integers greater than or equal to 2. Specifically, the values ​​of the first threshold and the second threshold can be flexibly designed by those skilled in the art based on actual conditions and are not limited.

[0111] In the present application, it is assumed that the first threshold is set to 3 and the second threshold is set to 3 respectively.

[0112] As mentioned above, there are 3 file hash values ​​corresponding to the fragment hash value #A1 (an example of the first quantity), namely file hash value #H1, file hash value #H2, and file hash value #H3. Among them, the open source code file corresponding to the file hash value #H1 is the open source code file #A, and there are 4 open source code warehouses to which the open source code file #A belongs (an example of the second quantity); the open source code file corresponding to the file hash value #H2 is the open source code file #B, and there are 3 open source code warehouses to which the open source code file #B belongs (another example of the second quantity); the open source code file corresponding to the file hash value #H3 is the open source code file #C, and there are 3 open source code warehouses to which the open source code file #C belongs (another example of the second quantity). Since the first quantity meets the first threshold and the second quantity meets the second threshold, the code fragment #A1 can be used as a public code fragment in the public code data warehouse.

[0113] As mentioned above, there are 2 file hash values ​​corresponding to the fragment hash value #A2 (another example of the first quantity), namely file hash value #H1 and file hash value #H3. Among them, the open source code file corresponding to the file hash value #H1 is the open source code file #A, and there are 4 open source code warehouses to which the open source code file #A belongs (an example of the second quantity), and the open source code file corresponding to the file hash value #H3 is the open source code file #C, and there are 3 open source code warehouses to which the open source code file #C belongs (another example of the second quantity). Since, although the first quantity meets the first threshold, the second quantity meets the second threshold, therefore, the code fragment #A2 can be used as a public code fragment in the public code data warehouse.

[0114] Furthermore, in order to more accurately screen out the common code snippets in the open source code, the following step 11 may be used for further screening.

[0115] Step 11: Determine the fourth mapping relationship.

[0116] A fourth mapping relationship is determined according to the file hash value and the first information. The fourth mapping relationship is a mapping relationship between the file hash value and the first information, wherein the first information includes the file name of the open source code file and / or the type of the file license of the open source code file.

[0117] Exemplarily, assume that the file names of the open source code file #A in different open source code repositories are different, for example, the file name in the open source code repositories #1 is file_name#a1, the file name in the open source code repositories #2 is file_name#a2, the file name in the open source code repositories #3 is file_name#a3, and the file name in the open source code repositories #4 is file_name#a4. Assume that the file names of the open source code file #B in different open source code repositories are different, for example, the file name in the open source code repositories #2 is file_name#b2, the file name in the open source code repositories #3 is file_name#b3, and the file name in the open source code repositories #4 is file_name#b4. Assume that the file names of the open source code file #C in different open source code repositories are different, for example, the file name in the open source code repositories #1 is file_name#c1, the file name in the open source code repositories #3 is file_name#c3, and the file name in the open source code repositories #4 is file_name#c4, as shown in Table 6 below.

[0118] Exemplarily, assume that the types of file licenses of open source code file #A in different open source code repositories are different, for example, the type of file license in open source code repository #1 is license#a1, the file name in open source code repository #2 is license#a2, the file name in open source code repository #3 is license#a3, and the file name in open source code repository #4 is license#a4. Assume that the file names of open source code file #B in different open source code repositories are different, for example, the file name in open source code repository #2 is license#b2, the file name in open source code repository #3 is license#b3, and the file name in open source code repository #4 is license#b4. Assume that the file names of open source code file #C in different open source code repositories are different, for example, the file name in open source code repository #1 is license#c1, the file name in open source code repository #3 is license#c3, and the file name in open source code repository #4 is license#c4, as shown in Table 6 below.

[0119] Table 6

[0120]

[0121] Step 12: According to the fourth mapping relationship, determine that the number of different file names of open source code files corresponding to each file hash value in the first quantity is a third number, and / or, according to the fourth mapping relationship, determine that the number of different types of file licenses of open source code files corresponding to each file hash value in the first quantity is a fourth number.

[0122] As shown in Table 6, there are three file hash values ​​corresponding to the fragment hash value #A1 (an example of the first number), namely file hash value #H1, file hash value #H2, and file hash value #H3. Among them, the open source code file corresponding to the file hash value #H1 is the open source code file #A, there are 4 open source code repositories to which the open source code file #A belongs (an example of the second quantity), there are 4 file names of the open source code file #A (an example of the third quantity), and there are 4 file license types of the open source code file #A (an example of the fourth quantity); the open source code file corresponding to the file hash value #H2 is the open source code file #B, there are 3 open source code repositories to which the open source code file #B belongs (an example of the second quantity), there are 3 file names of the open source code file #B (another example of the third quantity), and there are 3 file license types of the open source code file #B (an example of the fourth quantity); the open source code file corresponding to the file hash value #H3 is the open source code file #C, there are 3 open source code repositories to which the open source code file #C belongs (an example of the second quantity), there are 3 file names of the open source code file #C (another example of the third quantity), and there are 3 file license types of the open source code file #C (an example of the fourth quantity).

[0123] Step 13: When the first number is greater than or equal to the first threshold, and the second number is greater than or equal to the second threshold, and the third number is greater than or equal to the third threshold, and / or the fourth number is greater than or equal to the fourth threshold, the third code fragment corresponding to the same fragment hash value is determined as the second code fragment.

[0124] In the present application, the third threshold and the fourth threshold are both integers greater than or equal to 2. Specifically, the specific values ​​of the third threshold and the fourth threshold can be flexibly designed by those skilled in the art and are not limited.

[0125] Assume that, in this application, the third threshold is 2 and the fourth threshold is also 2.

[0126] As mentioned above, there are 3 file hash values ​​corresponding to code snippet #A1 (an example of the first number), the number of open source code repositories to which the open source code files corresponding to each of the 3 file hash values ​​belong is also greater than or equal to 3, and the number of file names of the open source code files corresponding to each of the 3 file hash values ​​is also greater than or equal to 3, and the number of file license types of the open source code files corresponding to each of the 3 file hash values ​​is also greater than or equal to 3. Therefore, code snippet #A1 can be used as a public code snippet in the public code database.

[0127] Optionally, 220, a common code database is generated based on multiple second code fragments.

[0128] It can be seen from the above step 210 that the second code snippets determined in the present application come from different open source code files and different open source code repositories, or in other words, multiple open source code files and multiple open source code repositories all include the same code snippet, that is, for some functions, developers will use the same code snippet to describe, so it can be considered as a common code snippet. A common code database can be constructed based on multiple second code snippets.

[0129] Further, each of the different open source code files corresponding to the second code snippet corresponds to at least two different file names, and / or each of the different open source code files corresponding to the second code snippet corresponds to at least two different file license types.

[0130] It can also be understood that, in this application, the essential reason why a second code snippet in the open source code can become a public code snippet in the public code database is precisely because the code snippet appears in multiple open source code files and multiple open source code repositories, and the open source code files in different open source code repositories correspond to different file names and different file license types. In other words, the same code snippet appears in different open source code files and open source code repositories, and the open source code files are presented in different open source code repositories with different file names and different file license types, which are essentially the same open source code files. Therefore, the code snippet can be identified as a public code snippet.

[0131] In the present application, the public code database includes at least two second code segments. Alternatively, it can also be understood that the public code database can be constructed by determining multiple second code segments.

[0132] 230. Obtain a first code snippet, where the first code snippet is a code snippet in a user code file that is similar to an open source code included in an open source code repository.

[0133] In the present application, the word "similar" in "a code snippet in a user code file that is similar to an open source code included in an open source code repository" can be understood as, for example, multiple consecutive lines of a code snippet in a user code file are similar or identical to multiple consecutive lines in a certain open source file included in the open source code repository.

[0134] In the present application, the number of code lines included in the first code snippet is N.

[0135] 240. If the public code database includes the first code segment, determine that the first code segment in the user code file is a public code segment.

[0136] As mentioned above, in the present application, the public code database includes multiple second code snippets, each of the multiple second code snippets belongs to at least two different open source code files, and each of the at least two different open source code files belongs to at least two different open source code repositories. In other words, in the present application, a second code snippet among the multiple second code snippets included in the public code database may be the first code snippet in the user code file.

[0137] In one possible implementation, for example, the code snippets in the user code file can be compared one by one with the code snippets in the public code database. If a code snippet in the public code database is exactly the same as a code snippet used in the open source code file, then it can be determined that the code snippet in the user code file is a public code snippet.

[0138] In another possible implementation, assuming that the number of code lines of each first code snippet in the user code file is 6, the fragment hash value corresponding to the first code snippet can be calculated first. It should be noted that the fragment hash values ​​calculated for the same code snippet are the same. Therefore, if the fragment hash values ​​corresponding to each of the multiple second code snippets included in the public code database include the fragment hash value corresponding to the first code snippet, the first code snippet is determined to be a public code snippet. For example, if the fragment hash value calculated by the first code snippet is also the fragment hash value #A1, the first code snippet is a public code snippet.

[0139] Figure 3 is a schematic diagram of a system architecture applicable to this application, and Figure 1 Compared with the system architecture shown, Figure 3The system architecture shown adds a "public code database module" and an "automatic confirmation service module". In this application, a public code database can be mined from open source code, so this application adds a "public code database module". The "automatic confirmation service module" can find public code snippets in the scan results and mark them as automatically confirmed results. Users can see the automatic confirmation results in the "confirmation service module" (for example, users can see that the type of automatic confirmation results is "public code snippets"), and can also manually modify the confirmation results if necessary.

[0140] Exemplarily, in this application, the scanning results obtained by the "scanning service module" include the path and line number range of the user code, the path and line number range of the matching open source code, and license information. The user code filtered by the "segment reference analysis engine module" is preprocessed in the same way as above, and then traversed with 6 lines as the window to obtain a list of fragment hash values ​​corresponding to the user code fragment; query whether the fragment hash value list of the user code fragment is included in the public code database, and if found, it means that the user code fragment is a public code fragment.

[0141] Furthermore, after finding all the public code snippets in the user code file, the lines corresponding to the public code snippets in the user code snippets filtered by the "Snippet Reference Analysis Engine Module" can be deleted. If the remaining code is less than 6 lines, it will be discarded, otherwise the remaining lines will be retained as matching snippets. Continuous public code snippets are stored in the database as a new matching snippet and marked as confirmed, with the confirmation type of "public code snippet"; users can see the confirmation results marked as public code on the interface, and can further see the source of the public code snippet, including the path and license information of each open source code. If the user thinks that the confirmation result of the "automatic confirmation module" has a problem, they can also receive a modified confirmation result.

[0142] In this application, due to the addition of an "automatic confirmation module", the computing device can automatically confirm whether the user's code snippet is a public code snippet. If the "snippet reference analysis engine module" includes some public code snippets in the user code file, the "automatic confirmation module" can automatically confirm it as a public code, and the line corresponding to the public code snippet can be deleted. The method provided by this application can reduce the false positive rate of user code snippet references screened in the existing "snippet reference analysis engine module", improve the accuracy of code detection and the efficiency of code development.

[0143] The communication method provided in the embodiment of the present application can be applied to the cloud service scenario, and the method is executed by the cloud management platform in the cloud service scenario. Figure 4 , describe the cloud service scenarios in detail.

[0144] Figure 4 is a schematic block diagram of a cloud scenario applicable to an embodiment of the present application. Figure 4 As shown, the cloud scenario may include: a cloud management platform 410 , the Internet 420 , and a client 430 .

[0145] like Figure 4 As shown, the cloud management platform 410 is used to manage the infrastructure that provides multiple cloud services. The infrastructure includes multiple cloud data centers, each of which includes multiple servers, and each server includes cloud service resources to provide corresponding cloud services to tenants. In the embodiment of the present application, the cloud service resources can be cloud databases.

[0146] The cloud management platform 410 may be located in a cloud data center, and it may provide an access interface (such as an interface or an application program interface (API)). The tenant may operate the client 430 to remotely access the access interface to register a cloud account and password on the cloud management platform 410, and log in to the cloud management platform 410. After the cloud management platform 410 successfully authenticates the cloud account and password, the tenant may further pay to select and purchase a virtual machine of specific specifications (processor, memory, disk) on the cloud management platform 410. After the paid purchase is successful, the cloud management platform 410 provides the remote login account and password of the purchased virtual machine, and the client 430 may remotely log in to the virtual machine, and install and run the tenant's application in the virtual machine. Therefore, the tenant may create, manage, log in and operate virtual machines in the cloud data center through the cloud management platform 410.

[0147] The functions of the cloud management platform 410 include, but are not limited to, a tenant console, computing management services, network management services, storage management services, authentication services, and image management services. The tenant console provides an interface or API to interact with tenants. The computing management service is used to manage servers running virtual machines and containers and bare metal servers. The network management service is used to manage network services (such as gateways, firewalls, etc.). The storage management service is used to manage storage services (such as data bucket services). The authentication service is used to manage tenant accounts and passwords, and the image management service is used to manage virtual machine images. Tenants can use the client 430 to log in to the cloud management platform 410 through the Internet 420 to manage the rented cloud services.

[0148] It can be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0149] Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0150] The embodiment of the present application can divide the computing device into functional modules according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0151] Figure 5 is a schematic block diagram of a computing device 500 provided in an embodiment of the present application. As shown in the figure, the computing device 500 may include: an acquisition module 510, a determination module 520. Optionally, it also includes a generation module 530.

[0152] Among them, the above-mentioned modules are respectively used to execute the steps of the above-mentioned methods, which will not be repeated here.

[0153] It should also be understood that the communication device 500 herein is embodied in the form of a functional unit. The term "unit" herein may refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0154] The computing device 500 of each of the above schemes has the function of implementing the corresponding steps of the above method 200. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the determination unit can be replaced by a processor to respectively perform the sending and receiving operations and related processing operations in each method embodiment. In addition, the determination unit can be a processing circuit.

[0155] In a possible implementation, the device 500 may be a computing device in the method 200 described above and applied in a cloud management platform, and the device is used to execute the actions in the method 200 described above.

[0156] It should be pointed out that Figure 5 The computing device in the above method embodiment may be the computing device, the cloud management platform (or the computing device in the cloud management platform), or the chip or chip system corresponding to the cloud management platform (or the computing device in the cloud management platform), for example, a system on chip (SoC). The acquisition module, the determination module, and the generation module are processors or microprocessors or integrated circuits integrated on the chip. This is not limited here.

[0157] Figure 6 6 is a schematic block diagram of another computing device 600 provided in an embodiment of the present application. As shown in the figure, the device 600 includes: at least one processor 620. The processor 620 is coupled to the memory and is used to execute instructions stored in the memory to send signals and / or receive signals. Optionally, the device 600 also includes a memory 630 for storing instructions. Optionally, the device 600 also includes a transceiver 610, and the processor 620 controls the transceiver 610 to send signals and / or receive signals.

[0158] It should be understood that the processor 620 and the memory 630 can be combined into one processing device, and the processor 620 is used to execute the program code stored in the memory 630 to implement the above functions. In specific implementation, the memory 630 can also be integrated into the processor 620, or independent of the processor 620.

[0159] It should also be understood that the transceiver 610 may include a transceiver (or receiver) and a transmitter (or transmitter). The transceiver may further include an antenna, and the number of antennas may be one or more. The transceiver 610 may also be a communication interface or an interface circuit.

[0160] Specifically, the processor 620 in the device 600 may correspond to the acquisition module 510 , the determination module 520 , and the generation module 530 in the device 500 .

[0161] As a solution, the device 600 is used to implement the steps corresponding to the computing device applied in the cloud management platform in the above method 200 embodiment.

[0162] As another solution, the device 600 is used to implement the steps corresponding to the cloud management platform in the above method 200 embodiment.

[0163] For example, the processor 620 is used to execute the computer program or instructions stored in the memory 630 to implement each step in the above method 200.

[0164] Figure 7 : is a schematic diagram of the architecture of a computing device cluster provided in an embodiment of the present application. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone. Figure 7 As shown, the computing device cluster includes at least one computing device 700. The memory 730 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the actions executed by the cloud management platform (or computing devices in the cloud management platform) in the above embodiment 200.

[0165] In some possible implementations, the memory 730 of one or more computing devices 7000 in the computing device cluster may also respectively store partial instructions for executing the actions performed by the cloud management platform (or, the communication device in the cloud management platform) in the method 200 described in the above embodiment. In other words, the combination of one or more computing devices 700 can jointly execute instructions for executing the actions performed by the cloud management platform (or, the communication device in the cloud management platform) described in the above embodiment.

[0166] It should be noted that the memory 730 in different computing devices 700 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the computing device 700. That is, the instructions stored in the memory 730 in different computing devices 700 may implement the functions of one or more modules of the determination module 510, the acquisition module 520 and the generation module 530.

[0167] Alternatively, the memory 730 in different computing devices 700 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the cloud management platform (or the communication devices in the cloud management platform) corresponding to the above communication devices 500 to 600. That is, the instructions stored in the memory 730 in different computing devices 700 may implement the functions of one or more modules in the determination module 510, the acquisition module 520, and the generation module 530.

[0168] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8As shown, two computing devices 700A and 700B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0169] It should be understood that Figure 8 The functions of the computing device 700A shown in FIG. 7 may also be completed by multiple computing devices 700. Similarly, the functions of the computing device 700B may also be completed by multiple computing devices 800.

[0170] In this embodiment, a computer program product including instructions is also provided, and the computer program product may be software or a program product including instructions that can be run on a computing device cluster or stored in any available medium. When the computer program product is run by a computing device cluster, the computing device cluster executes the method provided above, or the computing device cluster implements the function of the apparatus provided above.

[0171] In this embodiment, a computer-readable storage medium is also provided, and the computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk), etc. The computer-readable storage medium includes instructions, and when the instructions in the computer-readable storage medium are executed on a computing device, the computing device executes the method provided above.

[0172] In this embodiment, a computer-readable storage medium is also provided, and the computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc. The computer-readable storage medium includes instructions, and when the instructions in the computer-readable storage medium are executed by a computing device cluster, the computing device cluster executes the method provided above.

[0173] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0175] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0176] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0178] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0179] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for code detection and analysis, characterized in that: include: Obtain a first code snippet, where the first code snippet is a code snippet in a user code file that is similar to an open source code included in an open source code repository, wherein the number of code lines included in the first code snippet is N, and N is an integer greater than or equal to 2; If the first code snippet is included in a public code database, the first code snippet is determined to be a public code snippet, wherein the public code database includes multiple second code snippets, each of the multiple second code snippets belongs to at least two different open source code files, each of the at least two different open source code files belongs to at least two different open source code repositories, and the number of code lines included in each second code snippet is the N.

2. The method according to claim 1, characterized in that Each open source code file corresponds to at least two different file names, and / or each open source code file corresponds to at least two different types of file licenses.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: According to the user code file and the open source code repository, a code segment in the user code file that is similar to the open source code included in the open source code repository is determined.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In the open source code repository, determining the second code snippet; The common code database is generated according to a plurality of the second code fragments.

5. The method according to claim 4, characterized in that Each open source code file includes at least one third code snippet, and the number of code lines included in the third code snippet is N. The determining of the second code snippet comprises: Determine a file hash value corresponding to each of the open source code files according to the at least one third code snippet included in each of the code files; Determine a first mapping relationship according to each third code snippet in each open source code file and a file hash value corresponding to each open source code file, wherein the first mapping relationship is a mapping relationship between the third code snippet and the file hash value, wherein the file hash value is a file hash value corresponding to the open source code file to which the third code snippet belongs; Determine, according to the N lines of code included in each of the third code fragments, a fragment hash value corresponding to each of the third code fragments; Determine a second mapping relationship according to the first mapping relationship and the fragment hash value corresponding to each of the third code fragments, where the second mapping relationship is a mapping relationship between the fragment hash value corresponding to each of the third code fragments and the file hash value; According to the second mapping relationship, determine that the number of different file hash values ​​corresponding to the same segment hash value is a first number; Determine a third mapping relationship, where the third mapping relationship is a mapping relationship between the open source code file and the open source code repository to which it belongs; According to the third mapping relationship, determine that the number of different open source code repositories to which the open source code files corresponding to each file hash value in the different file hash values ​​in the first number belong is a second number; When the first number is greater than or equal to a first threshold, and / or the second number is greater than or equal to a second threshold, the third code fragment corresponding to the same fragment hash value is determined as the second code fragment, wherein the first threshold and the second threshold are both integers greater than or equal to 2.

6. The method according to claim 5, characterized in that The method further comprises: Determine a fourth mapping relationship according to the file hash value and the first information, where the fourth mapping relationship is a mapping relationship between the file hash value and the first information, wherein the first information includes a file name of the open source code file and / or a type of a file license of the open source code file; According to the fourth mapping relationship, determining that the number of different file names of the open source code files corresponding to each file hash value in the first quantity is a third number, and / or, according to the fourth mapping relationship, determining that the number of different types of file licenses of the open source code files corresponding to each file hash value in the first quantity is a fourth number; When the first number is greater than or equal to a first threshold, and / or the second number is greater than or equal to a second threshold, determining the third code fragment corresponding to the same fragment hash value as the second code fragment includes: When the first number is greater than or equal to a first threshold, and the second number is greater than or equal to a second threshold, and the third number is greater than or equal to a third threshold, and / or the fourth number is greater than or equal to a fourth threshold, the third code fragment corresponding to the same fragment hash value is determined as the second code fragment, wherein the third threshold and the fourth threshold are both integers greater than or equal to 2.

7. The method according to claim 6, characterized in that The method further comprises: Determine, according to the N lines of code included in the first code fragment, a fragment hash value corresponding to the first code fragment; If the public code database includes the first code snippet, determining that the first code snippet in the user code file is a public code snippet includes: If the fragment hash values ​​corresponding to each of the plurality of second code fragments included in the public code database include the fragment hash value corresponding to the first code fragment, then the first code fragment is determined to be the public code fragment.

8. The method according to any one of claims 1 to 7, characterized in that The method is executed by a cloud management platform.

9. A computing device, characterized in that include: Get module, determine module, where: The acquisition module is used to acquire a first code snippet, where the first code snippet is a code snippet in a user code file that is similar to an open source code included in an open source code repository, and the number of code lines included in the first code snippet is N, where N is an integer greater than or equal to 2; If the first code snippet is included in the public code database, the determination module is used to determine that the first code snippet in the user code file is a public code snippet, wherein the public code database includes multiple second code snippets, each of the multiple second code snippets belongs to at least two different open source code files, each of the at least two different open source code files belongs to at least two different open source code repositories, and the number of code lines included in each second code snippet is the N.

10. The computing device according to claim 9, characterized in that Each open source code file corresponds to at least two different file names, and / or each open source code file corresponds to at least two different types of file licenses.

11. The computing device according to claim 9 or 10, characterized in that: The determining unit is used to determine, based on the user code file and the open source code repository, a code segment in the user code file that is similar to the open source code included in the open source code repository.

12. The computing device according to any one of claims 9 to 11, characterized in that: The computing device further includes: a generating module, The determination module is used to determine the second code snippet in an open source code repository; The generating module is used to generate the common code database according to a plurality of the second code fragments.

13. The computing device according to claim 12, characterized in that Each open source code file includes at least one third code snippet, and the number of code lines included in the third code snippet is N. The determining module is used to determine the second code segment, including: The determination module is used to determine the file hash value corresponding to each open source code file according to the at least one third code fragment included in each code file; The determination module is used to determine a first mapping relationship according to each third code snippet in each open source code file and a file hash value corresponding to each open source code file, wherein the first mapping relationship is a mapping relationship between the third code snippet and the file hash value, wherein the file hash value is a file hash value corresponding to the open source code file to which the third code snippet belongs; The determination module is used to determine the fragment hash value corresponding to each of the third code fragments according to the N lines of code included in each of the third code fragments; The determination module is used to determine a second mapping relationship according to the first mapping relationship and the fragment hash value corresponding to each of the third code fragments, where the second mapping relationship is a mapping relationship between the fragment hash value corresponding to each of the third code fragments and the file hash value; The determining module is used to determine, according to the second mapping relationship, that the number of different file hash values ​​corresponding to the same segment hash value is a first number; The determination module is used to determine a third mapping relationship, where the third mapping relationship is a mapping relationship between the open source code file and the open source code repository to which it belongs; The determining module is used to determine, according to the third mapping relationship, that the number of different open source code repositories to which the open source code files corresponding to each file hash value in the different file hash values ​​in the first number belong is a second number; When the first number is greater than or equal to a first threshold, and / or the second number is greater than or equal to a second threshold, the determination module is used to determine the third code fragment corresponding to the same fragment hash value as the second code fragment, wherein the first threshold and the second threshold are both integers greater than or equal to 2.

14. The computing device according to claim 13, characterized in that The determination module is used to determine a fourth mapping relationship according to the file hash value and the first information, where the fourth mapping relationship is a mapping relationship between the file hash value and the first information, wherein the first information includes a file name of the open source code file and / or a type of a file license of the open source code file; According to the fourth mapping relationship, determining that the number of different file names of the open source code files corresponding to each file hash value in the first quantity is a third number, and / or, according to the fourth mapping relationship, determining that the number of different types of file licenses of the open source code files corresponding to each file hash value in the first quantity is a fourth number; When the first number is greater than or equal to a first threshold, and / or the second number is greater than or equal to a second threshold, the determining module is configured to determine the third code fragment corresponding to the same fragment hash value as the second code fragment, including: When the first number is greater than or equal to a first threshold, and the second number is greater than or equal to a second threshold, and the third number is greater than or equal to a third threshold, and / or the fourth number is greater than or equal to a fourth threshold, the determination module is used to determine the third code fragment corresponding to the same fragment hash value as the second code fragment, wherein the third threshold and the fourth threshold are both integers greater than or equal to 2.

15. The computing device according to claim 14, characterized in that The determining module is used to determine a fragment hash value corresponding to the first code fragment according to the N lines of code included in the first code fragment; If the public code database includes the first code snippet, the determining module is used to determine that the first code snippet in the user code file is a public code snippet, including: If the fragment hash values ​​corresponding to each of the plurality of second code fragments included in the public code database include the fragment hash value corresponding to the first code fragment, the determination module is used to determine that the first code fragment is the public code fragment.

16. The computing device according to any one of claims 9 to 15, characterized in that: The computing device is a cloud management platform, or the computing device is applied to a cloud management platform.

17. A computing device, characterized in that The method comprises a processor and a memory, wherein the processor runs instructions in the memory, so that the processor performs the method according to any one of claims 1 to 8.

18. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 8.

19. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 8.

20. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 8.

Images