Code detection method, device, medium and apparatus

Abstract

The application discloses a code detection method and device, a medium and equipment, relates to the field of XX, and the method comprises the following steps: acquiring source code to be detected and a preset word vector table corresponding to the source code; performing segmentation processing on the source code based on the syntax structure of the source code, to obtain a plurality of code segments; performing semantic extraction processing on each code segment in the plurality of code segments, to obtain semantic information of each code segment; determining semantic statistical feature data of each code segment based on the semantic information of each code segment and the preset word vector table; determining similar index data of each code segment pair in at least one code segment pair based on the semantic statistical feature data of each code segment; and determining at least one target code segment pair based on the similar index data of each code segment pair and a preset similarity condition. The semantic information and the semantic statistical feature data of the code segment can be used to more accurately and efficiently detect similar code segments in the source code.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to code detection methods, apparatus, media, and equipment. Background Technology

[0002] During the process of writing source code, the same function can be implemented repeatedly using multiple syntax structures, and the same logical judgment may have multiple equivalent representations, resulting in redundant parts in the source code, which affects the quality of the source code and the compilation and running performance.

[0003] Manually inspecting similar or duplicate code is extremely time-consuming and labor-intensive, especially in large software projects where the workload of code inspection and refactoring is enormous, which can severely slow down the software development process and reduce development efficiency. Furthermore, manual inspection often focuses only on the surface structure of the code, ignoring deeper semantic information, making it difficult to discover highly similar code with different surface structures. Summary of the Invention

[0004] To more efficiently and accurately detect similar or duplicate code in source code, this application provides a code detection method, apparatus, medium, and device. The technical solution is as follows:

[0005] Firstly, this application provides a code detection method, the method comprising:

[0006] Obtain the source code to be detected and the preset word vector table corresponding to the source code;

[0007] Based on the syntax structure of the source code, the source code is segmented to obtain multiple code fragments;

[0008] Semantic extraction processing is performed on each of the plurality of code fragments to obtain the semantic information of each code fragment;

[0009] Based on the semantic information of each code segment and the preset word vector table, the semantic statistical feature data of each code segment are determined;

[0010] Based on the semantic statistical feature data of each code segment, similarity index data of each code segment pair in at least one code segment pair is determined, wherein each code segment pair includes any two code segments from the plurality of code segments;

[0011] Based on the similarity index data and preset similarity conditions for each code snippet pair, at least one target code snippet pair is determined.

[0012] In one embodiment of this application, the method further includes:

[0013] From the preset refactoring rule base, determine the target refactoring rule corresponding to each target code segment pair in the at least one target code segment pair;

[0014] Based on the target reconstruction rules, the corresponding target code fragment pairs are reconstructed to obtain the target code fragments.

[0015] In one embodiment of this application, determining the target refactoring rule corresponding to each target code segment pair in the at least one target code segment pair from a preset refactoring rule base includes:

[0016] Obtain at least one reconstruction rule from the preset reconstruction rule base;

[0017] Based on the detection function of each of the at least one refactoring rules, the adaptation information between each refactoring rule and each target code fragment pair is determined;

[0018] Based on the adaptation information between each refactoring rule and each target code snippet pair, the target refactoring rule that is adapted to each target code snippet pair is determined.

[0019] In one embodiment of this application, the step of performing code refactoring processing on the corresponding target code segment pair based on the target refactoring rule to obtain the target code segment includes:

[0020] Based on the target reconstruction rules, the corresponding target code fragment pairs are merged to obtain the target code fragment;

[0021] Alternatively, based on the target reconstruction rule, the variable names in the corresponding target code fragment pair are modified to obtain the target code fragment.

[0022] In one embodiment of this application, the step of segmenting the source code based on its syntactic structure to obtain multiple code fragments includes:

[0023] Generate an abstract syntax tree corresponding to the source code; the abstract syntax tree indicates the syntactic structure of the source code;

[0024] A depth-first search is performed on the abstract syntax tree to obtain multiple syntax paths;

[0025] The source code is segmented according to the multiple syntax paths to obtain multiple code fragments, and the multiple code fragments correspond one-to-one with the multiple syntax paths.

[0026] In one embodiment of this application, each of the plurality of syntax paths contains at least one node in the abstract syntax tree, and the semantic extraction processing of each of the plurality of code fragments to obtain the semantic information of each code fragment includes:

[0027] Determine the node type of the at least one node contained in each syntax path;

[0028] Based on the node type, semantic extraction processing is performed on the code fragments corresponding to each syntax path to obtain the semantic information of each code fragment.

[0029] In one embodiment of this application, determining the semantic statistical feature data of each code segment based on the semantic information of each code segment and the preset word vector table includes:

[0030] Based on the semantic information of each code segment and the preset word vector table, a semantic vector for each code segment is generated; the preset word vector table includes at least one word vector.

[0031] Determine the statistical feature data of each word vector in the semantic vector of each code segment;

[0032] Based on the statistical feature data, the semantic statistical feature data of each code segment is determined.

[0033] In one embodiment of this application, determining the statistical feature data of each word vector in the semantic vector of each code segment includes:

[0034] Determine the word frequency of each word vector in the semantic vector of each code segment;

[0035] Determine the inverse document frequency of each word vector in a semantic vector set, the semantic vector set including the semantic vector of each code segment;

[0036] Based on the word frequency of each word vector in the semantic vector of each code segment and the inverse document frequency of each word vector in the set of semantic vectors, the statistical feature data of each word vector in the semantic vector of each code segment are determined.

[0037] In one embodiment of this application, the method further includes:

[0038] In response to a user-triggered detection command targeting the source code, the at least one pair of target code segments is determined;

[0039] Determine the detection result information and reconstruction guidance information corresponding to each target code segment pair in the at least one target code segment pair;

[0040] The detection results and reconstruction guidance information are displayed.

[0041] Secondly, this application provides a code detection device, the device comprising:

[0042] The acquisition module is used to acquire the source code to be detected and the preset word vector table corresponding to the source code;

[0043] The code segmentation module is used to segment the source code based on its syntactic structure to obtain multiple code fragments.

[0044] The semantic extraction module is used to perform semantic extraction processing on each of the multiple code segments to obtain the semantic information of each code segment;

[0045] The semantic statistical feature data determination module is used to determine the semantic statistical feature data of each code segment based on the semantic information of each code segment and the preset word vector table;

[0046] The similarity index data determination module is used to determine the similarity index data of each code segment pair in at least one code segment pair based on the semantic statistical feature data of each code segment, wherein each code segment pair includes any two code segments of the plurality of code segments;

[0047] The target code snippet pair filtering module is used to determine at least one target code snippet pair based on the similarity index data and preset similarity conditions of each code snippet pair.

[0048] Thirdly, this application provides a computer-readable storage medium storing at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the code detection method as described in the first aspect.

[0049] Fourthly, this application provides a computer device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the code detection method as described in the first aspect.

[0050] Fifthly, this application provides a computer program product, the computer program product including computer instructions, which, when executed by a processor, implement the code detection method as described in the first aspect.

[0051] The code detection method, apparatus, medium, and equipment provided in this application have the following technical effects:

[0052] The solution provided in this application firstly involves segmenting the source code based on its syntactic structure to obtain multiple code segments, which can be independent logical blocks. Semantic extraction is then performed on each of these segments to obtain semantic information. Based on this semantic information and a predefined word vector table, semantic statistical features of each segment can be determined. These features indicate the compositional or distributional characteristics of the semantic information of each segment. Furthermore, based on these features, similarity index data for each code segment pair can be determined. This similarity index data indicates the degree of similarity between the two code segments in terms of semantic composition or distribution. Finally, based on the similarity index data and predefined similarity conditions, at least one target code segment pair can be identified, where the similarity between the two code segments in each target pair satisfies the predefined similarity conditions. The solution provided in this application accurately describes the semantic composition or distribution characteristics of each code segment through semantic information extraction and semantic statistical feature data calculation. Based on the semantic statistical feature data of each code segment, duplicate or similar code segments can be found more comprehensively and accurately, improving the efficiency of automated detection and development, and making it suitable for large-scale applications. By reconstructing at least one pair of target code segments, the quality and execution performance of the source code can be effectively improved, as well as the readability and maintainability of the source code, reducing software development risks.

[0053] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0054] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic diagram of the implementation environment of a code detection method provided in an embodiment of this application;

[0056] Figure 2 This is a flowchart illustrating a code detection method provided in an embodiment of this application;

[0057] Figure 3This is a schematic diagram of a process for segmenting source code according to an embodiment of this application;

[0058] Figure 4 This is a flowchart illustrating a process for determining semantic statistical feature data for each code segment, as provided in an embodiment of this application.

[0059] Figure 5 This is a schematic diagram of another code detection process provided in an embodiment of this application;

[0060] Figure 6 This is a schematic diagram of an information display process based on a code detection method provided in an embodiment of this application;

[0061] Figure 7 This is a schematic diagram of a human-computer interaction process provided in an embodiment of this application;

[0062] Figure 8 This is a schematic diagram of a process provided in an embodiment of the present application, in which a terminal backend system executes an abstract syntax tree based on the source code to determine the semantic statistical feature data of the source code;

[0063] Figure 9 This is a schematic diagram of a code detection device provided in an embodiment of this application;

[0064] Figure 10 This is a schematic diagram of the hardware structure of a device for implementing a code detection method provided in an embodiment of this application. Detailed Implementation

[0065] To more efficiently and accurately detect similar or duplicate code in source code, embodiments of this application provide code detection methods, apparatus, media, and devices. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0067] Please see Figure 1 This is a schematic diagram of the implementation environment of a code detection method provided in an embodiment of this application, such as... Figure 1 As shown, the implementation environment can include at least client 01 and developer 02.

[0068] Specifically, the client 01 may include devices such as smartphones, desktop computers, tablets, laptops, in-vehicle terminals, digital assistants, smart wearable devices, and voice interaction devices. It may also include software running on the device, such as web pages provided to users by some service providers, or applications provided to users by these service providers. Specifically, the client 01 can obtain the source code to be detected input by the developer 02, and in response to the detection command triggered by the developer 02, determine the preset word vector table corresponding to the source code; the client 01 segments the source code based on its syntactic structure to obtain multiple code fragments; the client 01 performs semantic extraction processing on each of the multiple code fragments to obtain the semantic information of each code fragment; the client 01 determines the semantic statistical feature data of each code fragment based on the semantic information of each code fragment and the preset word vector table; the client 01 determines the similarity index data of each code fragment pair in at least one code fragment pair based on the semantic statistical feature data of each code fragment, where each code fragment pair includes any two code fragments from multiple code fragments; the client 01 determines at least one target code fragment pair based on the similarity index data of each code fragment pair and preset similarity conditions. The client 01 can display at least one target code fragment pair on the interface to prompt the developer 02 with the detection result information of the source code.

[0069] This application embodiment can also be implemented using cloud technology. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. It can also be understood as a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on cloud computing business models. Cloud technology requires cloud computing as its support. Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." Specifically, the client 01 can also communicate with the cloud server, send code detection tasks targeting source code to the cloud server, and receive at least one pair of target code fragments returned by the cloud server.

[0070] The following describes a code detection method provided in this application. Figure 2 This is a flowchart illustrating a code detection method provided in an embodiment of this application. This application provides the operational steps of the method described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or accompanying drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Please refer to... Figure 2 The code detection method provided in this application embodiment may include the following steps:

[0071] S210: Obtain the source code to be detected and the corresponding preset word vector table.

[0072] As is understandable, source code, also known as source program, is an uncompiled text file written according to certain programming language specifications. Source code consists of a series of readable computer language instructions.

[0073] In this embodiment of the application, the preset word vector table contains multiple words and multiple vector representations. The preset word vector table can indicate the mapping relationship between multiple words and multiple vector representations.

[0074] In one feasible implementation, the preset word vector table can be constructed based on the source code to be detected. Specifically, the source code is traversed, and words in each program statement are extracted, such as words representing attributes like object type, object name, function type, function name, interface name, parameter type, parameter name, variable type, variable name, identifier, operator, and structure name. Each word is then assigned an integer data type as its vector representation.

[0075] In one feasible implementation, the preset word vector table can be a general vocabulary, containing all words representing the types and attributes of grammar nodes and the vector representations corresponding to the words, wherein the grammar nodes can be lexical units, expressions, statements, function definitions, class definitions, and program structures, etc.

[0076] S220: Based on the syntax structure of the source code, the source code is segmented to obtain multiple code fragments.

[0077] In this application embodiment, the syntactic structure of the source code can be used to indicate how the source code is correctly parsed and executed. The syntactic structure of the source code can generally be divided into multiple levels, such as lexical units, expressions, statements, function / class definitions, and program structure. Lexical units are the smallest syntactic elements of the source code, such as keywords, identifiers, and operators; expressions are computational units composed of operators and operands, which can return a result value; statements are complete instructions that perform an action or control the program flow, usually ending with a newline or semicolon; function / class definitions are modular structures that encapsulate code logic; and program structure is a file or module-level structure, such as import declarations and package definitions.

[0078] In the embodiments of this application, the source code is segmented based on the functional modules, statements, expressions or other logical units indicated by the syntax structure of the source code, such as dividing the source code into independent functions, loop structures or conditional judgment blocks.

[0079] In one embodiment of this application, such as Figure 3 As shown, step S220 can be implemented as follows:

[0080] S221: Generate an abstract syntax tree corresponding to the source code; the abstract syntax tree indicates the syntactic structure of the source code.

[0081] Understandably, the Abstract Syntax Tree (AST) is a core data structure in compilers and interpreters, used to represent the syntactic structure of source code in a tree-like structure. The AST strips away the specific syntactic details of the source code (such as semicolons, parentheses, etc.), retaining only abstract nodes related to program logic, serving as a bridge for code analysis and transformation. Each node in the AST represents a syntactic structure, such as a function declaration, loop statement, variable assignment, expression, or function body.

[0082] For example, the source code for an addition function can be represented as follows:

[0083] function sum(a,b){

[0084] return a+b;

[0085] }

[0086] The corresponding abstract syntax tree can be represented as:

[0087]

[0088] Among them, the node FunctionDeclaration indicates a function declaration, Identifier is an identifier used to indicate variable names, function names, property names, etc., FormalParameters indicates formal parameters, BlockStatement indicates a block statement, which is a list of statements enclosed in a pair of curly braces, ReturnStatement indicates a return statement, and BinaryExpression indicates a binary expression, which is an expression consisting of two operands and one operator.

[0089] By generating an abstract syntax tree of the source code, it becomes easier to analyze and extract semantic information from the source code.

[0090] S222: Perform a depth-first search on the abstract syntax tree to obtain multiple syntax paths.

[0091] Among them, each syntax path obtained by performing a depth-first search (DFS) on the abstract syntax tree represents the access order from the root node of the abstract syntax tree to a certain leaf node. Multiple syntax paths can reflect the syntax structure hierarchy and logical nesting relationship of the source code.

[0092] Depth-first search is an algorithm used to traverse or search data structures such as trees and graphs. The main process is as follows: visit the starting node, which is the root node, and mark the starting node as the current node; recursively visit the unvisited neighboring nodes of the current node; after all the neighboring nodes of the current node have been visited, return to the parent node to continue the search.

[0093] S223: The source code is split according to multiple syntax paths to obtain multiple code fragments, and each code fragment corresponds one-to-one with a syntax path.

[0094] Specifically, each node in each syntax path represents a syntax structure, and each syntax path corresponds to a logical block in the source code. Thus, multiple code fragments corresponding one-to-one with multiple syntax paths can be obtained from the source code based on multiple syntax paths.

[0095] In another feasible implementation, the node types contained in the abstract syntax tree can be used to guide the segmentation of the source code. Function definition nodes, class definition nodes, loop statement nodes, conditional statement nodes, etc., are pre-defined as special node types. By identifying these special node types in the abstract syntax tree, the subtrees corresponding to these special node types can be determined. This allows the code fragments corresponding to the subtrees to be identified from the source code, ensuring the syntactic and semantic integrity of the code fragments.

[0096] In another feasible implementation, the syntax path can be directly used as a code fragment, and the corresponding code fragment can be represented by the node information of the nodes contained in the syntax path.

[0097] In the above embodiments, the source code is converted into an abstract syntax tree, and multiple code fragments are obtained as detection units based on the results of depth-first search. This can accurately reflect the logical boundaries of the code, especially suitable for nested structures (such as multi-level loops and conditional statements), and also facilitate the structured representation of code fragments, which is helpful for subsequent similarity detection and code refactoring.

[0098] In another embodiment of this application, a depth-first search can be used to traverse all nodes of the abstract syntax tree of the source code, recording the node type and other relevant information during the traversal. Taking the source code and abstract syntax tree of the example above as an example, the FunctionDeclaration node is visited first, followed by its child nodes such as Identifier and FormalParameters. Relevant information is extracted based on the node type. For example, the function name (sum) can be extracted from the FunctionDeclaration node, the parameter list (a, b) from the FormalParameters node, and the return value expression (a+b) from the ReturnStatement node, thus ensuring that key semantic information can be extracted. By vectorizing the extracted semantic information, a preset word vector table corresponding to the source code can be constructed. For example, the vocabulary can be represented as: {"FunctionDeclaration":0,"Identifier":1,"FunctionExpression":2,"AssignmentExpression":3,“BinaryExpression":4,"NumericLiteral":5}.

[0099] S230: Perform semantic extraction processing on each of the multiple code segments to obtain the semantic information of each code segment.

[0100] In one embodiment of this application, words are extracted from each code segment to obtain the semantic information of each code segment.

[0101] In one embodiment of this application, based on the embodiments provided in steps S221-S223, step S230 can be implemented as follows:

[0102] S231: Determine the node type of at least one node contained in each syntax path.

[0103] S232: Based on node type, perform semantic extraction processing on the code fragments corresponding to each syntax path to obtain the semantic information of each code fragment.

[0104] In one feasible implementation, information related to the node type is extracted based on the node type of each node, which serves as the semantic information of that node. The semantic information of each node contained in the syntax path constitutes the semantic information of the corresponding code segment, such as extracting the function name from the FunctionDeclaration node and extracting the parameter list from the FormalParameters node.

[0105] In another feasible implementation, specifically, function definition nodes, class definition nodes, loop statement nodes, conditional statement nodes, etc. are pre-defined as special types of nodes. Based on the node type and the pre-defined special types, at least one special node in the syntax path is determined, and the semantic information of each extracted special node is used to form the semantic information of the corresponding code segment.

[0106] In the above embodiments, by performing targeted semantic extraction based on the node type of each node in the syntax path, the semantics of each code fragment can be represented efficiently and accurately, improving the accuracy of code detection from the semantic analysis dimension.

[0107] S240: Based on the semantic information of each code segment and a preset word vector table, determine the semantic statistical feature data of each code segment.

[0108] In this embodiment of the application, based on the vector representation of each word in the preset word vector table, the semantic information of each code segment is converted into a word vector sequence, and the word vector sequence of each code segment is expanded to the same length. Words that do not appear in the semantic information of each code segment can be filled with 0 to facilitate the calculation of subsequent similarity indicators.

[0109] In the embodiments of this application, the semantic statistical feature data of each code segment can indicate the compositional or distributional features of the semantic information of each code segment.

[0110] In one embodiment of this application, such as Figure 4 As shown, step S240 can be implemented as follows:

[0111] S241: Generate a semantic vector for each code segment based on the semantic information of each code segment and a preset word vector table; the preset word vector table includes at least one word vector.

[0112] Specifically, based on at least one word vector included in the preset word vector table, where word vectors are vector representations of words, the vector representation corresponding to each word in the semantic information of each code segment can be determined, and the word vectors of each word constitute the semantic vector of the corresponding code segment.

[0113] S242: Determine statistical feature data for each word vector in the semantic vector of each code segment, in at least one word vector.

[0114] Specifically, each word vector in at least one word vector is traversed, and the statistical features of each word vector in the semantic vector of each code segment are determined. These statistical features can indicate the importance or discriminative power of each word vector in the semantic vector of each code segment.

[0115] In one feasible implementation, the statistical feature data of a word vector in the semantic vector of a code segment can be the product of the word frequency of the word vector in the code segment and the inverse document frequency of the word vector in the same code segment.

[0116] S243: Based on statistical feature data, determine the semantic statistical feature data for each code segment.

[0117] Specifically, based on the statistical feature data of each word vector in the semantic vector of each code segment and the positional order of each word vector in a preset word vector table, the semantic statistical feature data of each code segment can be constructed. The data type of the semantic statistical feature data of each code segment can be vector, and the semantic statistical feature data of each code segment includes the statistical feature data of each word vector in the semantic vector of this code segment.

[0118] In the above embodiments, by using a preset word vector table, the semantic information of each code segment is represented in a vectorized manner. This allows for the efficient calculation of the statistical feature data of each word vector in the semantic vector of each code segment, and the obtaining of the vectorized semantic statistical feature data of each code segment. This facilitates the efficient calculation of the similarity between code segments in the vector space.

[0119] In one embodiment of this application, the statistical feature data of a word vector in the semantic vector of a code segment can be the product of the word frequency of the word vector in the code segment and the inverse document frequency of the word vector in the same code segment. Then, step S242 can be implemented as follows:

[0120] S2421: Determine the word frequency of each word vector in the semantic vector of each code segment.

[0121] The word frequency of each word vector in the semantic vector of each code segment represents the frequency of occurrence of each word vector in each code segment. The higher the frequency of occurrence, the more important the word vector is in the code segment.

[0122] S2422: Determine the inverse document frequency of each word vector in the semantic vector set, which includes the semantic vector of each code snippet.

[0123] The inverse document frequency of each word vector in the semantic vector set can indicate the importance of each word vector in the semantic vector set.

[0124] For example, the formula for calculating Inverse Document Frequency (IDF) can be shown in formula (1):

[0125]

[0126] Where t represents word vectors, N represents the number of code snippets or semantic vectors, and n t This represents the number of semantic vectors containing t.

[0127] S2423: Determine the statistical feature data of each word vector in the semantic vector of each code segment based on the word frequency of each word vector in the semantic vector of each code segment and the inverse document frequency of each word vector in the set of semantic vectors.

[0128] Specifically, the word frequency of each word vector in the semantic vector of each code segment and the inverse document frequency of each word vector in the set of semantic vectors can be multiplied to obtain the statistical feature data of each word vector in the semantic vector of each code segment.

[0129] In the above embodiments, by calculating the product of the word frequency of each word vector in the semantic vector of each code segment and the inverse document frequency of each word vector, the composition and distribution characteristics of the semantic information of each code segment can be effectively indicated.

[0130] S250: Based on the semantic statistical feature data of each code segment, determine the similarity index data of each code segment pair in at least one code segment pair, wherein each code segment pair includes any two code segments from a plurality of code segments.

[0131] In the embodiments of this application, the data type of the semantic statistical feature data of each code segment can be a vector, and the semantic statistical feature data of each code segment includes the statistical feature data of each word vector in the semantic vector of this code segment.

[0132] In this embodiment, based on the semantic statistical feature data of each code segment, similarity indices are calculated pairwise to obtain similarity index data for each code segment pair. These similarity indices can be cosine similarity, Euclidean distance, Manhattan distance, etc. The similarity index data for each code segment pair indicates the degree of similarity in semantic composition and distribution between the two included code segments.

[0133] In one exemplary embodiment of this application, the following vocabulary is provided:

[0134] {"FunctionDeclaration":0,

[0135] "Identifier":1,

[0136] "FunctionExpression":2,

[0137] "AssignmentExpression":3,

[0138] “BinaryExpression”:4,

[0139] "NumericLiteral":5}

[0140] For two code snippets, the semantic information of each extracted code snippet is transformed according to the word vectors corresponding to each word in the vocabulary. The semantic vector of the first code snippet is represented as [0,1,2,1,4], and the semantic vector of the second code snippet is represented as [3,1,3,1,5].

[0141] (1) Calculate the term frequency (TF) of each word vector in the semantic vector of each code segment.

[0142] In the semantic vector of the first code snippet, the word frequency of word vector 0 is 1, the word frequency of word vector 1 is 2, the word frequency of word vector 2 is 1, the word frequency of word vector 3 is 1, the word frequency of word vector 4 is 1, and the word frequency of word vector 5 is 1. Therefore, the word frequency vector of the first code snippet can be represented as [1,2,1,0,1,0]. Similarly, the word frequency vector of the second code snippet can be represented as [0,2,0,2,0,1].

[0143] (2) Determine the inverse document frequency (IDF) of each word vector in the semantic vector set.

[0144] Based on formula (1), the following can be calculated:

[0145] The inverse document frequency of FunctionDeclaration is 0.301;

[0146] The inverse document frequency of the Identifier is 0;

[0147] The inverse document frequency of FunctionExpression is 0.301;

[0148] The inverse document frequency of AssignmentExpression is 0.301;

[0149] The inverse document frequency of BinaryExpression is 0.301;

[0150] The inverse document frequency of NumericLiteral is 0.301;

[0151] (3) Calculate the statistical feature data of each word vector in the semantic vector of each code segment.

[0152] In the first code snippet, the TF*IDF of FunctionDeclaration is 0.301;

[0153] In the first code snippet, the TF*IDF of the Identifier is 0;

[0154] In the first code snippet, the TF*IDF of FunctionExpression is 0.301;

[0155] In the first code snippet, the TF*IDF of AssignmentExpression is 0;

[0156] In the first code snippet, the TF*IDF of BinaryExpression is 0.301;

[0157] In the first code snippet, the TF*IDF of NumericLiteral is 0;

[0158] Combining the TF*IDF values ​​of each word vector into a single vector, the semantic statistical features of the first code snippet can be represented as [0.301,0,0.301,0,0.301,0].

[0159] In the second code snippet, the TF*IDF of FunctionDeclaration is 0;

[0160] In the second code snippet, the TF*IDF of the Identifier is 0;

[0161] In the second code snippet, the TF*IDF of FunctionExpression is 0;

[0162] In the second code snippet, the TF*IDF of AssignmentExpression is 0.301;

[0163] In the second code snippet, the TF*IDF of BinaryExpression is 0;

[0164] In the second code snippet, the TF*IDF of NumericLiteral is 0.301;

[0165] Combining the TF*IDF values ​​of each word vector into a single vector, the semantic statistical feature data of the second code snippet can be represented as [0,0,0,0.602,0,0.301].

[0166] (4) Calculate similarity

[0167] Cosine similarity is a metric for measuring the directional similarity between two non-zero vectors. It is used to calculate the cosine of the angle between the two vectors. Cosine similarity is mainly based on the angle between the two vectors. If the two vectors have the same direction, their cosine similarity is 1; if the two vectors have opposite directions, their cosine similarity is -1; if the two vectors have perpendicular directions, their cosine similarity is 0. In practical applications, negative values ​​can be ignored, and only the angle between the two vectors is considered. Therefore, the value of cosine similarity is usually between 0 and 1. For the semantic statistical feature data of the first code segment [0.301,0,0.301,0,0.301,0] and the semantic statistical feature data of the second code segment [0,0,0,0.602,0,0.301], it can be calculated according to formula (2):

[0168]

[0169] in, The dot product of two semantic statistical feature data. These are the magnitudes of the semantic statistical feature data of the first code snippet and the magnitudes of the semantic statistical feature data of the second code snippet, respectively.

[0170] S260: Based on the similarity index data and preset similarity conditions for each code snippet pair, determine at least one target code snippet pair.

[0171] In this embodiment of the application, the preset similarity condition is used to indicate the constraints that the similarity index data of two similar code segments must meet. The preset similarity condition may include a similarity index threshold.

[0172] In the embodiments of this application, at least one pair of target code fragments contains two similar code fragments in each pair of target code fragments.

[0173] In one embodiment of this application, after determining at least one pair of target code segments in the source code that satisfy preset similarity conditions, such as Figure 5 As shown, the above method may further include:

[0174] S270: From the preset refactoring rule base, determine the target refactoring rule corresponding to each target code segment pair in at least one target code segment pair.

[0175] The refactoring preset library includes at least one refactoring rule. For ease of management and application, at least one refactoring rule can be defined in a consistent format. A refactoring rule can be represented as an object and contains the following properties:

[0176] The name of the refactoring rule: a string data type used to describe the refactoring rule;

[0177] Description: String data used to explain the purpose and method of the refactoring rules.

[0178] Detection function: A function used to detect whether the current target code snippet pair meets the application conditions of this refactoring rule. The detection function can accept two arguments (the two code snippets in the target code snippet pair) and return a boolean value indicating whether the refactoring rule applies to both code snippets. The detection function can also accept one argument to refactor a single code snippet.

[0179] Application function: A function that applies refactoring rules to the current pair of target code snippets, performing refactoring on the current pair of target code snippets. The application function takes one or two arguments (indicating code snippets) and returns a new object containing the refactored code snippets.

[0180] For example, a reconstruction rule for extracting repeating functions can be expressed as:

[0181]

[0182]

[0183] For example, a refactoring rule for deleting unused variables and methods can be expressed as:

[0184] For example, a refactoring rule for renaming variables can be represented as:

[0185]

[0186]

[0187] In one feasible implementation, based on the properties of the above-mentioned reconstruction rules, step S270 can be implemented as follows:

[0188] S271: Obtain at least one reconstruction rule from the preset reconstruction rule base.

[0189] S272: Based on the detection function of each refactoring rule in at least one refactoring rule, determine the adaptation information between each refactoring rule and each target code snippet pair.

[0190] Specifically, the parameters of the two code segments contained in each target code segment pair are passed into the detection function contained in each reconstruction rule. Based on the output of the detection function contained in each reconstruction rule, the adaptation information between each reconstruction rule and each target code segment pair is determined. The adaptation information can be Boolean data. When the adaptation information is 0, it indicates that the reconstruction rule is not applicable to the target code band pair. When the adaptation information is 1, it indicates that the reconstruction rule is applicable to the target code band pair.

[0191] S273: Based on the adaptation information between each refactoring rule and each target code snippet pair, determine the target refactoring rule that is adapted to each target code snippet pair.

[0192] In the above embodiments, the detection function contained in each refactoring rule is used to determine the target code segment pair used by the corresponding refactoring rule. That is, a suitable refactoring rule can be selected for each target code segment pair so that accurate and feasible refactoring of each target code segment pair can be achieved.

[0193] S280: Based on the target refactoring rules, perform code refactoring on the corresponding target code fragment pairs to obtain the target code fragments.

[0194] In one embodiment of this application, based on the application functions contained in the target refactoring rules, code refactoring processing of the adapted target code fragment pairs is implemented to obtain the target code fragments, thereby reducing the overall redundancy of the source code and improving the quality and execution performance of the source code.

[0195] In one feasible implementation, the type of refactoring may include:

[0196] S281: Based on the target reconstruction rules, merge the corresponding target code fragment pairs to obtain the target code fragment.

[0197] When two code snippets in a target code snippet pair implement the same function, a single target code snippet can be obtained by merging them, such as extracting the duplicate function into a separate function.

[0198] S282: Alternatively, based on the target refactoring rules, modify the variable names in the corresponding target code snippet pair to obtain the target code snippet.

[0199] If the two code snippets in a target code snippet pair perform different functions, you can modify the variable name of one of the code snippets to obtain the corresponding target code snippet, while leaving the other code snippet unchanged.

[0200] In the above embodiments, by performing code refactoring on the target code fragments, the overall redundancy of the source code can be reduced, the quality and execution performance of the source code can be improved, and the readability of the source code can be enhanced, making it easier to maintain.

[0201] In one embodiment of this application, after determining at least one pair of target code segments in the source code that satisfy preset similarity conditions, such as Figure 6 As shown, the above method may further include:

[0202] S291: In response to a user-triggered inspection instruction targeting source code, identify at least one pair of target code segments.

[0203] The process of identifying at least one target code segment pair in the source code can be referred to the foregoing embodiments, and will not be repeated here.

[0204] S292: Determine the detection result information and reconstruction guidance information corresponding to each target code segment pair in at least one target code segment pair.

[0205] Specifically, by combining a comprehensive refactoring rule base with the detection results information corresponding to each target code snippet, specific and practical refactoring guidance information can be automatically generated. The refactoring guidance information provides refactoring suggestions for the target code snippet pair, such as modifying variable names or deleting useless variables.

[0206] S293: Display test results and reconstruction guidance information.

[0207] For example, the detection results information and reconstruction guidance information can be displayed as follows:

[0208] Similarity: 85.7%

[0209] Code snippet 1: / path / to / module1.py:12

[0210] Code snippet 2: / path / to / module2.py:34

[0211] Suggested fix: It can be integrated into a single function.

[0212] In the above embodiments, the code detection methods provided in this application can be integrated into an application tool. Developers only need to provide source code and trigger detection commands to obtain detection results and refactoring guidance, effectively helping developers optimize code structure efficiently. Simultaneously, the detection results and refactoring guidance are displayed through an intuitive interface, providing convenient interactive functions, allowing developers to easily view detailed analysis and implement refactoring suggestions, thereby optimizing the entire software development process.

[0213] In one embodiment of this application, the above-described code detection method can also be used to detect two source code files, and the corresponding detection results and refactoring guidance information can be displayed as follows:

[0214] Similar source code: Source code 1 and Source code 2

[0215] Similarity: 85.7%

[0216] Code snippet 1 from source code 1: / path / to / module1.py:12

[0217] Code snippet 2 from source code 2: / path / to / module2.py:34

[0218] Refactoring suggestion: It can be integrated into a single function.

[0219] Similarity: 72.3%

[0220] Code snippet 3 from source code 1: / path / to / module3.py:45

[0221] Code snippet from source code 2: 4 / path / to / module4.py:67|

[0222] Refactoring suggestion: It can be integrated into a single function.

[0223] Overall similarity score: 78%.

[0224] Figure 7 A flowchart illustrating a code detection method provided in an embodiment of this application for a developer user is shown, as follows: Figure 7 As shown, developers only need to provide the source code and trigger a detection command to obtain detection results and refactoring guidance information, which includes suggestions for refactoring the source code. Developers can also trigger automatic refactoring of the source code based on the displayed refactoring guidance information to obtain optimized source code, or choose to ignore the refactoring guidance information based on their own experience.

[0225] Figure 8This document illustrates a flowchart illustrating the process of a terminal backend system determining semantic statistical feature data of source code based on an abstract syntax tree (AST). First, a depth-first search is performed on the AST to obtain multiple syntax paths. The source code is then segmented according to these paths, resulting in multiple code fragments. Semantic extraction is performed on each code fragment to obtain its semantic information. Based on the semantic information of each code fragment and a pre-defined word vector table, a semantic vector for each code fragment is generated. The statistical feature data of each word vector within the semantic vector of each code fragment is then calculated using the pre-defined word vector table, thereby obtaining the vectorized semantic statistical feature data of each code fragment. The specific process can be found in the aforementioned embodiments and will not be elaborated upon here.

[0226] As can be seen from the above embodiments, in the code detection method provided by this application, the source code can first be segmented based on the syntax structure of the source code to obtain multiple code segments, which can be independent logical blocks; semantic extraction processing is performed on each of the multiple code segments to obtain the semantic information of each code segment, and based on the semantic information of each code segment and a preset word vector table, the semantic statistical feature data of each code segment can be determined. The semantic statistical feature data can indicate the composition features or distribution features of the semantic information of each code segment; based on the semantic statistical feature data of each code segment, the similarity index data of each code segment pair in at least one code segment pair can be determined. The similarity index data of each code segment pair can indicate the degree of similarity between the two code segments contained therein in the semantic composition or distribution dimension; thus, based on the similarity index data of each code segment pair and a preset similarity condition, at least one target code segment pair can be determined, and the degree of similarity between the two code segments contained in each target code segment pair satisfies the preset similarity condition. The solution provided in this application accurately describes the semantic composition or distribution characteristics of each code segment through semantic information extraction and semantic statistical feature data calculation. Based on the semantic statistical feature data of each code segment, duplicate or similar code segments can be found more comprehensively and accurately, improving the efficiency of automated detection and development, and making it suitable for large-scale applications. By reconstructing at least one pair of target code segments, the quality and execution performance of the source code can be effectively improved, as well as the readability and maintainability of the source code, reducing software development risks.

[0227] This application embodiment also provides a code detection device 900, such as Figure 9 As shown, the device 900 may include:

[0228] The acquisition module 910 is used to acquire the source code to be detected and the preset word vector table corresponding to the source code;

[0229] The code segmentation module 920 is used to segment the source code based on its syntactic structure to obtain multiple code fragments.

[0230] The semantic extraction module 930 is used to perform semantic extraction processing on each of the plurality of code segments to obtain the semantic information of each code segment;

[0231] The semantic statistical feature data determination module 940 is used to determine the semantic statistical feature data of each code segment based on the semantic information of each code segment and the preset word vector table;

[0232] The similarity index data determination module 950 is used to determine the similarity index data of each code segment pair in at least one code segment pair based on the semantic statistical feature data of each code segment, wherein each code segment pair includes any two code segments of the plurality of code segments;

[0233] The target code snippet pair filtering module 960 is used to determine at least one target code snippet pair based on the similarity index data and preset similarity conditions of each code snippet pair.

[0234] In one embodiment of this application, the device 900 may further include:

[0235] The target reconstruction rule determination unit is used to determine the target reconstruction rule corresponding to each target code segment pair in the at least one target code segment pair from a preset reconstruction rule library;

[0236] The code refactoring unit is used to perform code refactoring processing on the corresponding target code segment pairs based on the target refactoring rules to obtain the target code segment.

[0237] In one embodiment of this application, the target reconstruction rule determination unit includes:

[0238] The reconstruction rule acquisition subunit is used to acquire at least one reconstruction rule from the preset reconstruction rule library;

[0239] The adaptation information determination subunit is used to determine the adaptation information between each reconstruction rule and each target code segment pair based on the detection function of each reconstruction rule in the at least one reconstruction rule;

[0240] The target refactoring rule determination subunit is used to determine the target refactoring rule that is adapted to each target code segment pair based on the adaptation information of each refactoring rule and each target code segment pair.

[0241] In one embodiment of this application, the code refactoring unit includes:

[0242] Based on the target reconstruction rules, the corresponding target code fragment pairs are merged to obtain the target code fragment;

[0243] Alternatively, based on the target reconstruction rule, the variable names in the corresponding target code fragment pair are modified to obtain the target code fragment.

[0244] In one embodiment of this application, the code splitting module 920 includes:

[0245] A syntax tree generation unit is used to generate an abstract syntax tree corresponding to the source code; the abstract syntax tree indicates the syntax structure of the source code;

[0246] The path search unit is used to perform a depth-first search on the abstract syntax tree to obtain multiple syntax paths;

[0247] The code segmentation unit is used to segment the source code according to the multiple syntax paths to obtain multiple code fragments, and the multiple code fragments correspond one-to-one with the multiple syntax paths.

[0248] In one embodiment of this application, the semantic extraction module 930 includes:

[0249] A node type determination unit is used to determine the node type of the at least one node contained in each syntax path;

[0250] The semantic information extraction unit is used to perform semantic extraction processing on the code fragments corresponding to each syntax path based on the node type, so as to obtain the semantic information of each code fragment.

[0251] In one embodiment of this application, the semantic statistical feature data determination module 940 includes:

[0252] A semantic vector generation unit is used to generate a semantic vector for each code segment based on the semantic information of each code segment and the preset word vector table; the preset word vector table includes at least one word vector.

[0253] A statistical feature data determination unit is used to determine the statistical feature data of each word vector in the semantic vector of each code segment;

[0254] The semantic statistical feature data determination unit is used to determine the semantic statistical feature data of each code segment based on the statistical feature data.

[0255] In one embodiment of this application, the statistical feature data determination unit includes:

[0256] A word frequency determination unit is used to determine the word frequency of each word vector in the semantic vector of each code segment;

[0257] An inverse document frequency determination unit is used to determine the inverse document frequency of each word vector in a semantic vector set, wherein the semantic vector set includes the semantic vector of each code segment;

[0258] The statistical feature data determination unit is used to determine the statistical feature data of each word vector in the semantic vector of each code segment based on the word frequency of each word vector in the semantic vector of each code segment and the inverse document frequency of each word vector in the set of semantic vectors.

[0259] In one embodiment of this application, the device 900 further includes:

[0260] The target code fragment pair determination unit is configured to determine the at least one target code fragment pair in response to a user-triggered detection instruction for the source code.

[0261] The information determination unit is used to determine the detection result information and reconstruction guidance information corresponding to each target code segment pair in the at least one target code segment pair;

[0262] The information display unit is used to display the detection result information and the reconstruction guidance information.

[0263] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0264] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0265] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement a code detection method as provided in the above method embodiments.

[0266] Figure 10 A schematic diagram of the hardware structure of a device for implementing a code detection method provided in an embodiment of this application is shown. This device may constitute or include the apparatus or system provided in the embodiment of this application. Figure 10 As shown, device 10 may include one or more processors 1002 (shown as 1002a, 1002b, ..., 1002n in the figure) 1002 (processor 1002 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 10 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 10 may also include a... Figure 10 The more or fewer components shown, or having the same Figure 10 The different configurations shown.

[0267] It should be noted that the aforementioned one or more processors 1002 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within device 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0268] The memory 1004 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, thereby implementing the above-mentioned code detection method. The memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include memory remotely located relative to the processor 1002, and these remote memories can be connected to the device 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0269] The transmission device 1006 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 10. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1006 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0270] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of device 10 (or a mobile device).

[0271] This application embodiment also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing a code detection method in the method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the code detection method provided in the above method embodiment.

[0272] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0273] This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a code detection method provided in the various optional embodiments described above.

[0274] As can be seen from the embodiments of the code detection method, apparatus, medium, and device provided in this application, the technical solution provided in this application firstly segments the source code based on its syntactic structure to obtain multiple code segments, which can be independent logical blocks; for each of the multiple code segments, semantic extraction is performed to obtain the semantic information of each code segment; and based on the semantic information of each code segment and a preset word vector table, the semantic statistical feature data of each code segment can be determined, which can indicate the composition or distribution features of the semantic information of each code segment; based on the semantic statistical feature data of each code segment, the similarity index data of each code segment pair in at least one code segment pair can be determined, which can indicate the degree of similarity between the two code segments contained in the semantic composition or distribution dimension; thus, based on the similarity index data of each code segment pair and a preset similarity condition, at least one target code segment pair can be determined, and the degree of similarity between the two code segments contained in each target code segment pair satisfies the preset similarity condition. The solution provided in this application accurately describes the semantic composition or distribution characteristics of each code segment through semantic information extraction and semantic statistical feature data calculation. Based on the semantic statistical feature data of each code segment, duplicate or similar code segments can be found more comprehensively and accurately, improving the efficiency of automated detection and development, and making it suitable for large-scale applications. By reconstructing at least one pair of target code segments, the quality and execution performance of the source code can be effectively improved, as well as the readability and maintainability of the source code, reducing software development risks.

[0275] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0276] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0277] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0278] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A code detection method, characterized in that, The method includes: Obtain the source code to be detected and the preset word vector table corresponding to the source code; Based on the syntax structure of the source code, the source code is segmented to obtain multiple code fragments; Semantic extraction processing is performed on each of the plurality of code fragments to obtain the semantic information of each code fragment; Based on the semantic information of each code segment and the preset word vector table, the semantic statistical feature data of each code segment are determined; Based on the semantic statistical feature data of each code segment, similarity index data of each code segment pair in at least one code segment pair is determined, wherein each code segment pair includes any two code segments from the plurality of code segments; Based on the similarity index data and preset similarity conditions for each code snippet pair, at least one target code snippet pair is determined.

2. The method according to claim 1, characterized in that, The method further includes: From the preset refactoring rule base, determine the target refactoring rule corresponding to each target code segment pair in the at least one target code segment pair; Based on the target reconstruction rules, the corresponding target code fragment pairs are reconstructed to obtain the target code fragments.

3. The method according to claim 2, characterized in that, The step of determining the target reconstruction rule corresponding to each target code segment pair in the at least one target code segment pair from the preset reconstruction rule base includes: Obtain at least one reconstruction rule from the preset reconstruction rule base; Based on the detection function of each of the at least one refactoring rules, the adaptation information between each refactoring rule and each target code fragment pair is determined; Based on the adaptation information between each refactoring rule and each target code snippet pair, the target refactoring rule that is adapted to each target code snippet pair is determined.

4. The method according to claim 2, characterized in that, The step of refactoring the corresponding target code fragment pairs based on the target refactoring rules to obtain target code fragments includes: Based on the target reconstruction rules, the corresponding target code fragment pairs are merged to obtain the target code fragment; Alternatively, based on the target reconstruction rule, the variable names in the corresponding target code fragment pair are modified to obtain the target code fragment.

5. The method according to claim 1, characterized in that, Based on the syntax structure of the source code, the source code is segmented to obtain multiple code fragments, including: Generate an abstract syntax tree corresponding to the source code; the abstract syntax tree indicates the syntactic structure of the source code; A depth-first search is performed on the abstract syntax tree to obtain multiple syntax paths; The source code is segmented according to the multiple syntax paths to obtain multiple code fragments, and the multiple code fragments correspond one-to-one with the multiple syntax paths.

6. The method according to claim 5, characterized in that, Each of the multiple syntax paths contains at least one node in the abstract syntax tree. The semantic extraction processing of each of the multiple code segments to obtain the semantic information of each code segment includes: Determine the node type of the at least one node contained in each syntax path; Based on the node type, semantic extraction processing is performed on the code fragments corresponding to each syntax path to obtain the semantic information of each code fragment.

7. The method according to claim 1, characterized in that, The step of determining the semantic statistical feature data of each code segment based on the semantic information of each code segment and the preset word vector table includes: Based on the semantic information of each code segment and the preset word vector table, a semantic vector for each code segment is generated; the preset word vector table includes at least one word vector. Determine the statistical feature data of each word vector in the semantic vector of each code segment; Based on the statistical feature data, the semantic statistical feature data of each code segment is determined.

8. The method according to claim 7, characterized in that, The step of determining the statistical feature data of each word vector in the semantic vector of each code segment includes: Determine the word frequency of each word vector in the semantic vector of each code segment; Determine the inverse document frequency of each word vector in a semantic vector set, the semantic vector set including the semantic vector of each code segment; Based on the word frequency of each word vector in the semantic vector of each code segment and the inverse document frequency of each word vector in the set of semantic vectors, the statistical feature data of each word vector in the semantic vector of each code segment are determined.

9. The method according to claim 1, characterized in that, The method further includes: In response to a user-triggered detection command targeting the source code, the at least one pair of target code segments is determined; Determine the detection result information and reconstruction guidance information corresponding to each target code segment pair in the at least one target code segment pair; The detection results and reconstruction guidance information are displayed.

10. A code detection device, characterized in that, The device includes: The acquisition module is used to acquire the source code to be detected and the preset word vector table corresponding to the source code; The code segmentation module is used to segment the source code based on its syntactic structure to obtain multiple code fragments. The semantic extraction module is used to perform semantic extraction processing on each of the multiple code segments to obtain the semantic information of each code segment; The semantic statistical feature data determination module is used to determine the semantic statistical feature data of each code segment based on the semantic information of each code segment and the preset word vector table; The similarity index data determination module is used to determine the similarity index data of each code segment pair in at least one code segment pair based on the semantic statistical feature data of each code segment, wherein each code segment pair includes any two code segments of the plurality of code segments; The target code snippet pair filtering module is used to determine at least one target code snippet pair based on the similarity index data and preset similarity conditions of each code snippet pair.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the code detection method as described in any one of claims 1 to 9.

12. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the code detection method as described in any one of claims 1 to 9.

Images