Code tracking method and system

Abstract

The invention relates to a code tracking method and system.The method comprises the steps that a source code file and a code submission record are obtained, the code submission record comprises code submission information and code change content, and the code change content and the source code file have an incidence relation; constructing a corresponding first syntax tree structure for each source code file, wherein the first syntax tree structure comprises a plurality of syntax sub-trees; splitting the code change contents according to the association relationship, constructing a second syntax tree structure for each code change content, and labeling code submission information; and performing similarity matching on the plurality of syntax sub-trees in the second syntax tree structure and the first syntax tree structure, and merging the second syntax tree structure into the first syntax tree structure according to a matching result. The system is used for executing the corresponding method steps. And by using the constructed code tracking syntax tree, rapid tracing of code defects is realized, the code quality risk in a project is further reduced, and the software quality is improved.

Description

Technical Field

[0001] This application primarily relates to the field of software development, and in particular to a method and system for code tracing. Background Technology

[0002] As the scope of cyberattacks continues to expand and the number and complexity of various application systems increase simultaneously, the reliable operation of application systems no longer solely depends on the meticulous analysis and rational design of technical solutions, but is also closely related to defect prevention in the system coding implementation stage. To reduce the operational risks of application systems and minimize potential defects in software code writing, many R&D teams have introduced source code scanning tools into the entire software development process. This assists developers in quickly finding, locating, fixing, and managing software code defects throughout their entire lifecycle.

[0003] In real-world R&D scenarios, most large-scale application systems are developed collaboratively by multi-person teams. Code issues detected by source code scanning tools not only require precise identification of the corresponding author but also necessitate a thorough investigation of whether similar defects exist in the code written by that individual in other projects. However, with the continuous iteration and upgrading of R&D projects, the size of R&D teams is increasing, and the collaborative code development process is becoming increasingly complex. Traditional defect handling methods are no longer sufficient to meet the demands of efficient management. Therefore, there is an urgent need to propose an efficient method for code defect tracking and investigation. Summary of the Invention

[0004] One objective of this application is to provide a method and system for code tracing, in order to solve the problems in the prior art where code defects are difficult to trace and cannot be efficiently investigated.

[0005] According to one aspect of this application, a method for code tracing is provided. The method includes: obtaining source code files and code commit records, wherein the code commit records include code commit information and code change content, and the code change content is associated with the source code files; constructing a corresponding first syntax tree structure for each source code file, wherein the first syntax tree structure includes multiple syntax subtrees; splitting the code change content according to the association, constructing a second syntax tree structure for each code change content, and marking the code commit information in the second syntax tree structure; performing similarity matching between the second syntax tree structure and the multiple syntax subtrees in the first syntax tree structure, and merging the second syntax tree structure into the first syntax tree structure according to the matching result for code tracing.

[0006] Optionally, the step of performing similarity matching between the second syntax tree structure and multiple syntax subtrees in the first syntax tree structure includes: generating a first feature vector based on the node type frequency of each syntax subtree in the first syntax tree structure, and generating a second feature vector based on the node type frequency of the second syntax tree structure; constructing a hash table based on the first feature vector, determining a hash signature based on the second feature vector, determining a candidate set of syntax subtrees corresponding to the first syntax tree structure based on the hash table and the hash signature; determining the cosine similarity between the second feature vector and the first feature vector corresponding to each syntax subtree in the candidate set of syntax subtrees, and comparing the cosine similarity with a set threshold to complete the similarity matching process.

[0007] Optionally, merging the second syntax tree structure into the first syntax tree structure based on the matching result includes: when the cosine similarity is greater than the set threshold, replacing the corresponding syntax subtree in the first syntax tree structure with the second syntax tree structure; when the cosine similarity is less than the set threshold, inserting the second syntax tree structure as a new syntax subtree into the first syntax tree structure.

[0008] Optionally, marking the code commit information in the second syntax tree structure includes: finding the syntax subtree containing the code change content in the second syntax tree structure, and marking the code commit information at the root node of the syntax subtree.

[0009] Optionally, the method further includes: obtaining the issue code, converting the issue code into a syntax tree structure to be matched; determining the first syntax tree structure of the source code file corresponding to the issue code; and locating the code submission information corresponding to the issue code based on the syntax tree structure to be matched and the first syntax tree structure.

[0010] Optionally, locating the code submission information corresponding to the problematic code based on the syntax tree structure to be matched and the first syntax tree structure includes: determining the similarity between each syntax subtree in the first syntax tree structure and the syntax tree structure to be matched; determining a target similar syntax subtree based on the similarity; and obtaining the code submission information carried by the target similar syntax subtree to locate the code submission information corresponding to the problematic code.

[0011] Optionally, the code submission information includes submitting user identification information, and the method further includes: obtaining the submitting user's historical code submission records based on the submitting user identification information; determining the similarity between the historical code submission records and the problematic code; and sending a warning message to the submitting user based on the similarity and a preset threshold.

[0012] According to another aspect of this application, a code tracing system is provided, comprising: a data acquisition module, a feature extraction module, and a matching update module; the data acquisition module is used to acquire source code files and code commit records, wherein the code commit records include code commit information and code change content, and the code change content is associated with the source code files; the feature extraction module is used to construct a corresponding first syntax tree structure for each source code file, wherein the first syntax tree structure includes multiple syntax subtrees; the feature extraction module is further used to split the code change content according to the association relationship, and construct a second syntax tree structure for each code change content, wherein the second syntax tree structure carries code commit information; the matching update module is used to perform similarity matching between the second syntax tree structure and multiple syntax subtrees in the first syntax tree structure, and merge the second syntax tree structure into the first syntax tree structure according to the matching result.

[0013] According to another aspect of this application, an electronic device is also provided, the electronic device comprising: one or more processors; and a memory storing computer-readable instructions, which, when executed, cause the processor to perform the operations of any of the methods described above.

[0014] According to another aspect of this application, a computer-readable storage medium is also provided, having stored thereon computer instructions that can be executed by a processor to implement the steps of any of the methods described above.

[0015] Compared with existing technologies, this application enables rapid source tracing of code defects by constructing an abstract syntax tree for code tracing, and efficiently checks whether other code has similar problems, thereby further reducing code quality risks and improving software quality in projects. Attached Figure Description

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 A flowchart illustrating a code tracing method according to one aspect of this application is shown; Figure 2 This illustration shows a schematic diagram of code commit record splitting in one embodiment of this application; Figure 3 This diagram illustrates feature extraction of code change content in one embodiment of this application; Figure 4 This diagram illustrates the first syntax tree structure update and merging in one embodiment of this application; Figure 5This illustration shows a diagram of code submission information annotated at the root node of a subtree in one embodiment of this application; Figure 6 A flowchart illustrating a problem code tracing method in one embodiment of this application is shown; Figure 7 This diagram illustrates a system block diagram for code tracing according to another aspect of this application; Figure 8 This diagram illustrates the overall technical architecture of code tracing in one embodiment of this application. Figure 9 A schematic diagram of a frame of an electronic device provided according to another aspect of this application is shown.

[0017] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0018] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein, and therefore this application is not limited to the specific embodiments disclosed below.

[0020] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0021] Figure 1 The diagram shows a flowchart of a code tracing method according to one aspect of this application, the method comprising steps S11 to S14.

[0022] Step S11: Obtain the source code file and code commit record, wherein the code commit record includes code commit information and code change content, and the code change content is related to the source code file.

[0023] Source code files are the core carriers of an application system, written by contributing users such as developers and R&D personnel. When developers modify and submit their code, a full record is generated, namely the code commit record, which includes code commit information and code changes. The code commit information records relevant developer information, such as developer account, name, commit time, commit notes (e.g., "Fixed the null pointer defect in the login interface"), and commit ID. The code change information records the details of the modified code, such as adding a function, deleting a piece of redundant code, or modifying conditional logic. However, the analysis granularity of code change content is the entire method body (Function), rather than isolated single lines of code, thus avoiding focusing on single-line modifications while ignoring the overall logic.

[0024] The code changes in a single code commit record usually include changes to multiple source code files. Each part of the code change content has a one-to-one correspondence with a specific source code file, and the code change content can be accurately matched to a specific source code file.

[0025] Step S12: Construct a corresponding first syntax tree structure for each source code file, wherein the first syntax tree structure includes multiple syntax subtrees.

[0026] After obtaining all source code files, a standardized syntax analysis process is used to generate a unique structured syntax tree for each source code file, namely the first syntax tree structure. This tree is then split into multiple independent substructures, namely multiple syntax subtrees, according to the logical structure of the source code file, thereby transforming the scattered source code text into a tree structure.

[0027] Specifically, this can be implemented using the Abstract Syntax Tree (AST) algorithm. An AST is a tree-like representation of the abstract syntactic structure of source code, where each node represents a structure within the source code. It is described as "abstract" because it does not represent every detail of the actual syntactic structure.

[0028] Lexical analysis breaks down the source code text into meaningful basic units, such as keywords, variable names, and operators, removing meaningless content (such as spaces and comments). Then, through syntax analysis, these basic units are organized into an abstract tree structure according to the syntax rules of the programming language. This transforms the source code file into a corresponding AST (Abstract Syntax Tree), resulting in a unique AST for each source code file.

[0029] Step S13: The code change content is split according to the association relationship, a second syntax tree structure is constructed for each code change content, and the code submission information is marked in the second syntax tree structure.

[0030] Since the code changes in a single code commit usually include changes to multiple source files, and each part of the code change content has a one-to-one correspondence with a specific source file, the code change content is split according to the correspondence. For example, the code change content can be split into changes belonging to file A, changes belonging to file B, etc.

[0031] exist Figure 2 In the illustrated embodiment, the left side shows the original code commit record, including a unique code commit identifier, the submitting user's email address, the code changes made, the scope of the code changes, and the commit time. The code changes include multiple source code files, such as TestClass01 and TestClass02. Splitting the code changes according to the source code files yields two separate code commit records on the right: one for the TestClass01 source code file and the other for the TestClass02 source code file.

[0032] For the code changes after splitting, the AST (Abstract Syntax Tree) algorithm can be used again for feature extraction and structured representation to generate a dedicated second syntax tree structure. Figure 3 In the illustrated embodiment, after feature extraction and representation of the code change content "public String method(...){...}" on the left, the code change content presented on the right in the form of an AST syntax tree structure is obtained: {type: "MethodDecl", id: "methodA", modifiers: "public", returnType: "String", ...}.

[0033] Each second syntax tree uniquely corresponds to a clearly defined segment of code changes and is associated with a specific source file. Furthermore, each second syntax tree structure is labeled with the code commit information tags corresponding to this code change, such as the committer, commit time, and commit comments. This not only accurately reflects "what was modified" but also enables complete association of traceable information such as "who made the change and when."

[0034] Step S14: Perform similarity matching between the second syntax tree structure and multiple syntax subtrees in the first syntax tree structure, and merge the second syntax tree structure into the first syntax tree structure according to the matching results.

[0035] The similarity of the second syntax tree corresponding to the code changes with each subtree in the first syntax tree corresponding to the source code file is compared. Based on the comparison results, such as whether the similarity exceeds a threshold, the matching subtree in the first syntax tree is replaced with the second syntax tree, or the second syntax tree is inserted as a new subtree into the first syntax tree. Finally, the second syntax tree is merged into the first syntax tree, realizing a structured record of code changes.

[0036] In one embodiment of the application, in step S14, a first feature vector is generated based on the node type frequency of each subtree in the first syntax tree structure, and a second feature vector is generated based on the node type frequency of the second syntax tree structure; a hash table is constructed based on the first feature vector, a hash signature is determined based on the second feature vector, and a candidate set of syntax subtrees corresponding to the first syntax tree structure is determined based on the hash table and the hash signature; the cosine similarity between the second feature vector and the first feature vector corresponding to each subtree in the candidate set of syntax subtrees is determined, and the cosine similarity is compared with a set threshold to complete the similarity matching process.

[0037] For ease of description and understanding, the second syntax tree structure is called Tree A, and the syntax tree of its parent source file, i.e., the first syntax tree structure, is called Tree B. In fact, Tree A is a subtree of Tree B. The most similar subtree of Tree A can be found in Tree B using feature vectors and the Locality-sensitive Hashing (LSH) algorithm. Suppose there is a Java source file Example.java, whose current syntax tree (Tree B) already contains a method; the developer ming@email.com submitted a code change modifying the method body, therefore this change (Tree A) needs to be merged into Tree B.

[0038] For example, the original method subtree in tree B (denoted as subtree B1) is: `public String greet(String userName) { String msg = "Hello," + userName; return msg;}`; while the modified tree A (denoted as subtree A) submitted by the developers is: `public String greet(String userName) { String msg = "Welcome," + userName + "!"; return msg;}`; the change is to change the string concatenation from "Hello," + userName to "Welcome," + userName + "!".

[0039] The first step is to generate feature vectors for the frequency of node types for both the A-tree and B-tree. A "node type frequency histogram" can be used as the feature vector. First, a dictionary of all possible node types needs to be defined. Assume the AST parser defines the following node types and assigns them indices: Index 0 corresponds to MethodDecl, Index 1 corresponds to VarDecl, Index 2 corresponds to BinaryExpr (binary expressions, such as assignment =, string concatenation +), Index 3 corresponds to LiteralExpr (literal expressions, such as strings and numbers), Index 4 corresponds to ReturnStmt, Index 5 corresponds to Identifier (identifiers, such as variable names), and Index 6 corresponds to Parameter. There are other node types as well.

[0040] Next, feature vectors are generated for the subtree. Each node in the subtree is traversed, the frequency of each node type is counted, and then a feature vector V is generated for the modified tree A (subtree A). A For example, the simplified structure of subtree A is rooted at MethodDecl, containing modifiers (public), returnType (String), id (greet), Parameter (userName), and Block (method body). The Block node contains two child nodes: VarDecl and ReturnStmt. The VarDecl node's child nodes are type (String), id (msg), and init (BinaryExpr(+), corresponding to "Welcome," +userName). The init node's child nodes are further divided into left (LiteralExpr ("Welcome,")) and right (Identifier (userName)). The ReturnStmt node's child node is Identifier (msg).

[0041] The frequency of node types in subtree A is calculated as follows: MethodDecl: 1, VarDecl: 1, BinaryExpr: 1, LiteralExpr: 1, ReturnStmt: 1, Identifier: 2, Parameter: 1. Other node types have a frequency of 0. Based on this, the feature vector V is generated. A It is a high-dimensional sparse vector such as [1, 1, 1, 1, 1, 2, 1, 0, 0, ...].

[0042] Similarly, generate feature vector V for the subtree to be compared in the B-tree (e.g., subtree B1). B1Since the AST of the original method `greet` is highly similar to the A-tree structure, differing only in literals, the frequency of node types can be calculated as follows: MethodDecl: 1, VarDecl: 1, BinaryExpr: 1, LiteralExpr: 1, ReturnStmt: 1, Identifier: 2, Parameter: 1, with other types of nodes having 0 nodes. Based on this, the generated feature vector V... B1 [1, 1, 1, 1, 1, 2, 1, 0, 0, ...].

[0043] It is important to note that V A and V B1 In this example, they are completely equal because the change did not alter the node type structure of the AST, but only changed the value of the literal, thus enabling the location of the change to be located through structural similarity.

[0044] Cosine similarity is used to measure V. A and V B1 The similarity between two feature vectors is calculated using the formula: Similarity ,in Let ||V|| denote the dot product, and ||V|| denote the magnitude (Euclidean length) of the vector. The eigenvector V of subtree A... A The eigenvector V of subtree B1 B1 For example, the dot product of the two (Main Dimension Contribution), Modulus || V A || = sqrt(1^2 + 1^2 + ... + 2^2 + ...) ≈ 3.32, modulus ||V B1 The cosine similarity between the two is approximately 3.32, which gives a cosine similarity of approximately equal to... .

[0045] The calculated cosine similarity is close to 1, indicating that the two subtrees are highly similar in AST structure. However, in scenarios where a B-tree contains hundreds or thousands of methods (i.e., subtrees), calculating the cosine similarity with all subtrees for each change is extremely inefficient. In such cases, the Locality Sensitive Hashing (LSH) algorithm can be used to quickly find similarities like V. B1 Such candidates.

[0046] The specific steps are divided into two stages: preprocessing and querying. In the preprocessing stage, an LSH index needs to be built for all subtrees of the B-tree. First, subtree enumeration and vectorization are performed, i.e., traversing the B-tree to generate a feature vector V for each meaningful subtree unit (such as a subtree rooted at each MethodDecl or IfStmt). iNext, random hyperplanes are generated. Three random hyperplanes (using the LSH function) are selected to generate a 3-bit signature. Each hyperplane is represented by a random normal vector r. j Define (j=1,2,3), and assume r j The dimension of the normal vector is the same as that of the feature vector, and each component takes a random value of 1 or -1. For example, the three generated random normal vectors are (simplified): r1 = [1, -1, 1, -1, 1, -1, 1, ...]; r2 = [-1, 1, -1, 1, -1, 1, -1, ...]; r3 = [1, 1, -1, -1, 1, 1, -1, ...].

[0047] Next, the hash signature (bit string) is calculated for the vector V of each subtree. i Calculate its relationship with the normal vector r of each random hyperplane. j dot product h j = V i r j If h j If h is 0, then the j-th bit of the signature is 1; if h j If the j-th bit is 0, then the j-th bit of the signature is 0, resulting in a 3-bit binary signature (e.g., 101).

[0048] With subtree B1 (V B1 Taking (e.g.,) as an example, we can calculate h1 = V B1 r1 = S1, assuming S1 If 0, then bit 1 = 1; h2 = V B1 r2 = S2, assuming S2 < 0, then bit 2 = 0; h3 = V B1 r3 = S3. Assuming S3 >= 0, then bit 3 = 1. The corresponding signature is 101. Finally, a hash table is constructed, that is, the pointers or IDs of all subtrees with signature 101 (including B1) are put into the "bucket" with label 101. This process is repeated for all subtrees in the B-tree to establish a mapping from "signature" to "subtree list".

[0049] During the query phase, the goal is to find the most similar subtree for modifying the A-tree. First, a query vector is generated, which is the previously obtained V. A Then, using the same three random hyperplanes r1, r2, r3 as in the preprocessing stage, let V A Calculate the signature. Since V A = V B1Therefore, the corresponding query signature is calculated to be 101. Then, the bucket with signature 101 is searched in the LSH hash table. Since V A and V B1 The vectors are almost identical, and they have a high probability (guaranteed by the theory of LSH) that they will be projected onto the same signature. Therefore, a small candidate set containing a few subtrees such as B1 can be found immediately from a massive number of subtrees.

[0050] Within this small candidate set (which may only have a few subtrees), V is precisely calculated using the cosine similarity formula. A The similarity between B1 and the vector of each subtree in the candidate set is calculated. Finally, the result is returned. If B1 has the highest similarity to A, which is 0.91 and exceeds a preset threshold (such as 0.8), then B1 is determined to be the "most similar subtree" of A in the B tree. Subsequently, the old B1 subtree in the B tree is replaced with a new A tree labeled with ming@email.com and the ID of this submission, thus completing the corresponding update process.

[0051] Abstract syntax trees (ASTs) transform the global search problem into a "search within a specific hash bucket" problem by mapping high-dimensional feature vectors to short bit strings (signatures), thus achieving efficient approximate nearest neighbor search. This enables rapid location of changes within the syntax tree of massive source code files, and is a key technology supporting the efficiency of this method.

[0052] Furthermore, when the cosine similarity is greater than the set threshold, the corresponding subtree in the first syntax tree structure is replaced by the second syntax tree structure; when the cosine similarity is less than the set threshold, the second syntax tree structure is inserted into the first syntax tree structure as a new subtree.

[0053] Following the above embodiment, the cosine similarity between the second syntax tree structure A and the syntax subtree B1 in the first syntax tree structure B is 0.91, which is greater than the set threshold of 0.8. Therefore, B1 is the most similar subtree of tree A in tree B, and tree A is used to directly replace subtree B1 in tree B. If the cosine similarity between tree A and all syntax subtrees in the first syntax tree structure B is less than the set threshold of 0.8, then no subtree similar to tree A can be found in tree B. In this case, tree A is added as a new branch of tree B and inserted into the corresponding position in tree B as a new subtree structure.

[0054] exist Figure 4In the illustrated embodiment, the first syntax tree structure, B-tree, is the baseline syntax tree corresponding to the initial source code file. It contains the complete structure of the original code, such as nodes like the method definition `MethodDecl` and the return statement `ReturnStmt`, and marks the code commit information of different contributors for different parts of the source code file each time. The second syntax tree structure, A-tree, is the syntax tree corresponding to the code changes in this instance. The "ReturnStmt" node has been modified to accommodate the new change logic that needs to be integrated into B-tree. Based on the similarity judgment between the syntax subtrees of A-tree and B-tree, the most similar subtree "ReturnStmt" of A-tree was found in B-tree. Therefore, A-tree is used as the new ReturnStmt subtree to replace the original ReturnStmt subtree in B-tree. The right side shows the updated B-tree, which retains the original core structure of the first syntax tree and includes the changes from the second syntax tree. Additionally, the root node `ReturnStmt` of the subtree marks the information of the contributors who made the changes, namely the submitter's email address `ming@email.com` and the unique commit identifier `b0014fa7`.

[0055] In one embodiment of this application, the syntax subtree containing the code change content is located in the second syntax tree structure, and the code submission information is marked on the root node of the syntax subtree.

[0056] The second syntax tree is a structured tree representation of the code changes. For example, a method modification corresponds to a syntax tree, but this tree can still be broken down into multiple syntax subtrees, such as the method definition subtree and the conditional statement subtree. Within the second syntax tree, the subtree containing the actual code change is located. For instance, if a conditional statement within a method is modified, the syntax tree corresponding to that conditional statement is located; if the entire method definition is modified, the syntax tree corresponding to that method is located.

[0057] After locating the target syntax subtree, code commit information is only annotated on the root node of that subtree. The root node is the top-level node of the subtree and is the core identifier of this logical unit. Annotating the root node ensures that the annotation position is unique and will not be repeatedly annotated; in addition, during subsequent searches, only the root node needs to be checked to quickly associate the traceability information of this change unit, without having to traverse the entire subtree. By binding the code commit information to key positions in the second syntax tree, the precise correspondence between traceability information and change logic is ensured.

[0058] exist Figure 5In the illustrated embodiment, the submitting user (email address: ming@email.com) modified the initial source code file and submitted it, with the unique submission identifier being b0014fa7. First, the code changes corresponding to this submission are obtained, and their features are extracted and structured represented using an AST (Abstract Syntax Tree). The changes are then converted into a corresponding AST structure, i.e., a second syntax tree specific to this code change. Within the generated second syntax tree, the actual syntax subtree corresponding to the code changes is precisely located, representing the logical unit related to this modification. On the root node "ReturnStmt" (return statement node) of this syntax subtree, the core information corresponding to this submission is marked: the submitting user's email address "ming@email.com" and the unique submission identifier "b0014fa7".

[0059] exist Figure 6 The illustrated embodiment shows a flowchart of a problem code tracing method, which includes steps S21 to S23.

[0060] Step S21: Obtain the problematic code and convert it into a syntax tree structure to be matched. The code tracing method can be used to trace the source of problematic code. First, find the code segment with defects or problems, i.e., the problematic code. Then, use the AST algorithm to convert this text-based problematic code into a structured syntax tree structure to be matched, thereby transforming the unstructured problematic code text into a tree structure that can be used for comparative analysis.

[0061] Step S22: Determine the first syntax tree structure of the source code file corresponding to the problematic code. Based on the source code file to which the problematic code belongs, generate the complete AST syntax tree corresponding to that source code file, i.e., the first syntax tree structure, which is the base syntax tree of that source code file.

[0062] Step S23: Locate the code submission information corresponding to the problematic code based on the syntax tree structure to be matched and the first syntax tree structure. Match the syntax tree structure to be matched, i.e., the tree representation of the problematic code, with the first syntax tree structure to find the corresponding subtrees, and then associate them with the code submission information marked in that subtree, such as the submitter and submission identifier. Through precise syntax tree matching, find the relevant personnel and submission records who wrote or modified this problematic code, thereby achieving precise source tracing of the problematic code.

[0063] In one embodiment of this application, in step S23, the similarity between each subtree in the first syntax tree structure and the syntax tree structure to be matched is determined, and a target similar syntax subtree is determined based on the similarity; the code submission information carried by the target similar syntax subtree is obtained to locate the code submission information corresponding to the problematic code.

[0064] First, traverse each subtree in the first syntax tree structure, such as the method subtree and conditional statement subtree, and calculate their similarity to the syntax tree structure to be matched, such as cosine similarity. From all the syntax subtrees, select the subtree with the highest similarity that exceeds a set threshold, and determine it as the target similar syntax subtree. That is, the code logic corresponding to this subtree highly matches the core logic of the problem code.

[0065] The system reads all code submission information marked on the root node of the target similar syntax subtree, such as submitter, submission time, and unique submission identifier. The extracted information is the submission information corresponding to writing or modifying the problematic code, thus completing the tracing chain of "problematic code → similar syntax subtree → submission information". By using structured syntax tree matching to replace traditional text comparison, the accuracy and efficiency of locating problematic code submission information are greatly improved.

[0066] Furthermore, the code submission information includes submitting user identification information; the submitting user's historical code submission records are obtained based on the submitting user identification information; the similarity between the historical code submission records and the problematic code is determined; and a warning message is sent to the submitting user based on the similarity and a preset threshold.

[0067] From the located code commit information, extract key identifiers, such as the submitter's identification information (email / employee ID), submission time, and unique submitter identifier, which can be used as auxiliary identifiers. Based on these identifiers, retrieve the user's historical code commit records across all projects, including all code changes made in the past and their corresponding syntax trees. Calculate the similarity between the syntax tree corresponding to the user's historical commit records (second syntax tree) and the syntax tree to be matched with the current problematic code, thereby identifying whether the user has written content in other code that has similar logic to the problematic code. For example, if user Zhang San's problematic code is a "null pointer defect," and if 3 out of 10 of his historical code commits have a similarity exceeding the threshold to the problematic code, it indicates that these 3 commits may also have the same null pointer defect problem.

[0068] The calculated similarity is compared with a preset threshold. If the similarity is greater than the preset threshold, an alert is sent to the submitting user. This proactively reminds the user to check for similar defects in their historical code, rather than just fixing a single problem that is currently discovered, thereby reducing the overall defect risk of the user's code from the source.

[0069] Figure 7The diagram illustrates a code tracing system block diagram according to another aspect of this application. The system includes: a data acquisition module 10, a feature extraction module 20, and a matching update module 30. The data acquisition module 10 is used to acquire source code files and code commit records, wherein the code commit records include code commit information and code change content, and the code change content is associated with the source code files. The feature extraction module 20 is used to construct a corresponding first syntax tree structure for each source code file, wherein the first syntax tree structure includes multiple syntax subtrees. The feature extraction module 20 is further used to split the code change content according to the association relationship and construct a second syntax tree structure for each code change content, wherein the second syntax tree structure carries code commit information. The matching update module 30 is used to perform similarity matching between the second syntax tree structure and multiple syntax subtrees in the first syntax tree structure, and merge the second syntax tree structure into the first syntax tree structure according to the matching result.

[0070] The data acquisition module 10 is used to collect basic data, obtaining source code files in the R&D scenario, as well as code commit records (including commit information and change content) associated with these files, providing data for subsequent analysis. The feature extraction module 20 is used for structured transformation. On the one hand, it generates a corresponding first syntax tree (split into multiple subtrees) for each source code file. On the other hand, it splits the code change content and generates a second syntax tree with commit information for each change, converting text-based code and changes into an analyzable tree structure. The matching and updating module 30 is used to merge syntax trees. It performs similarity matching between the second syntax tree and the subtrees of the first syntax tree, and merges the second syntax tree into the first syntax tree based on the results, realizing the structured recording and updating of code changes.

[0071] In code issue tracing and source identification scenarios, commit records from all projects' Git repositories are collected, features are extracted, and the latest representation structure of each source file in each project is stored in a database as an AST (Abstract Syntax Tree). Each key tree node of each complete representation structure (i.e., a complete source file) is labeled with its corresponding developer, indicating that this node and its subtree nodes were created by that developer, or that the most recent update was performed by that developer. Popular open-source tools such as OSV-Scanner, Findbugs, and SonarQube are used to scan and detect code defects. When a specific code issue is encountered in a specific project, the features of the problematic code are extracted to quickly locate its position in the source file's syntax tree, thus identifying the corresponding developer (i.e., the developer who wrote the problematic code). Based on that developer, the syntax trees of all source files that this developer participated in developing are identified, and tree similarity calculations are used to quickly check these source files to confirm whether the same problematic code exists.

[0072] exist Figure 8 The illustrated embodiment demonstrates the overall technical architecture of code tracing, including Git code repository, source code feature collection, source code scanning tools, source code matching analysis, and code scanning management.

[0073] Git is a distributed version control system built on the Linux kernel and is currently the most mainstream version control system. It can effectively and quickly handle version management for projects ranging from very small to very large. A Git service is installed and set up on a remote server. The development team uses Git to store project source code and manage versions, and developers collaborate through it. During daily development, after logging in and authenticating, developers submit changes to the code to the Git service through the code commit portal. The Git service automatically generates a corresponding commit record for each code change.

[0074] After developers complete code modifications, they commit the changes to a remote Git repository. Git automatically generates commit records (including the committer's email address, code changes / scope, commit time, etc.). The Git repository also bears the core responsibilities of source code storage and version control. After collecting commit records from the Git repository, the source code files are transformed into an Abstract Syntax Tree (AST), and the corresponding commit information is marked in the AST. Each source code file corresponds to a unique AST structure, bound to traceability information such as the committer and time, providing a structured foundation for subsequent defect analysis.

[0075] The code scanning management module calls the APIs of source code scanning tools (such as OSV-Scanner, Findbugs, and SonarQube) to periodically trigger scans of Git source code. After completing static or dynamic analysis, the source code scanning tools output code problem data (including details of the problematic code, its location in the file, and the type of defect), which is then stored in the database by the code scanning management module. Code defects refer to various potential pitfalls existing in program code. These pitfalls may lead to problems such as unauthorized access, data leakage, and system crashes. Common code defects include input validation vulnerabilities, cross-site scripting (XSS) attacks, SQL injection, buffer overflows, code injection, session hijacking, and password leakage.

[0076] Extract the AST (Abstract Syntax Tree) features of the problematic code and match them with the syntax trees of the corresponding source code files in the database (using algorithms such as LSH) to trace the source code back to the developer who submitted it. Read all ASTs submitted by that developer throughout their history, calculate the similarity with the problematic code's syntax tree, and determine if similar defects exist in their past code. Send alerts to the corresponding developers for all suspected problematic code snippets.

[0077] From the moment developers submit code, through Git storage, syntax tree construction, code scanning, and matching analysis, a closed loop is ultimately achieved: "automatic defect detection → tracing the responsible party → identifying similar issues → precise early warning," effectively managing code defect risks.

[0078] The proposed solution uses Git and source code scanning tools as its underlying support, and leverages AST (Abstract Syntax Tree) to achieve feature extraction and structured representation of source code. It also designs and implements functional modules such as source code feature collection, source code matching, and code scanning management. Its core lies in recording source code content and change iteration information through AST, accurately marking the origin of each line of code. This not only clearly identifies the source of code defects but also quickly checks for similar problems in other code, thus achieving efficient code defect tracking.

[0079] Compared to existing technologies, the proposed solution offers advantages in multiple scenarios: In collaborative environments with numerous R&D projects and large teams, a single scan of each project is sufficient to target and investigate suspicious projects and confirm similar defects, significantly improving the efficiency of defect detection and tracking; during project iterations, the syntax tree and change node markings clearly record the code's change and iteration trajectory, accurately pinpointing the source of each line of code and enhancing the accuracy and control of code version management; and in large, complex R&D scenarios, the responsibility for vulnerable code can be clearly defined, allowing problematic code to be directly pushed to the actual developers. Since developers are most familiar with the code logic, this approach also effectively improves the efficiency of fixing problematic code.

[0080] Figure 9 The diagram shows a schematic frame of an electronic device according to another aspect of this application, the electronic device including at least a processor 901 and a memory 902.

[0081] Processor 901 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 901 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 901 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 901 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 901 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0082] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 902 are used to store at least one instruction, which is executed by the processor 901 to implement a code tracing method provided in the method embodiments of this application.

[0083] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 901, memory 902, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuits, touch displays, audio circuits, and power supplies.

[0084] Of course, the electronic device may also include fewer or more components, and this embodiment does not limit this.

[0085] This application also provides a computer-readable storage medium storing computer instructions thereon, which can be executed by a processor to implement a code tracing method as described above.

[0086] When a code tracing method is implemented as a computer program, it can also be stored as an article of art in a computer-readable storage medium. For example, computer-readable storage media can include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes), optical discs (e.g., compact discs (CDs), digital multifunction discs (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memory (EPROM), cards, sticks, key drives). Furthermore, the various storage media described herein can represent one or more devices and / or other machine-readable media used for storing information. The term "machine-readable medium" can include, but is not limited to, wireless channels and various other media (and / or storage media) capable of storing, containing, and / or carrying code and / or instructions and / or data.

[0087] It should be understood that the embodiments described above are merely illustrative. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor may be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or combinations thereof.

[0088] Some aspects of this application can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The aforementioned hardware or software may be referred to as a "data block," "module," "engine," "unit," "component," or "system." The processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. Furthermore, aspects of this application may manifest as computer products residing in one or more computer-readable media, including computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), optical discs (e.g., compressed CDs, digital multifunction DVDs, etc.), smart cards, and flash memory devices (e.g., cards, sticks, key drives, etc.).

[0089] A computer-readable medium may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and so on, or suitable combinations thereof. A computer-readable medium can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer-readable medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, radio frequency signals, or similar media, or any combination of the above media.

[0090] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the exemplary embodiments of this application.

[0091] Furthermore, this application uses specific terms to describe embodiments of the application. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of the application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of the application can be appropriately combined.

[0092] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of scope in some embodiments of this application are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

Claims

1. A code tracing method, characterized in that, The method includes: Obtain source code files and code commit records, wherein the code commit records include code commit information and code change content, and the code change content is associated with the source code files; For each source code file, a corresponding first syntax tree structure is constructed, wherein the first syntax tree structure includes multiple syntax subtrees; The code change content is split according to the association relationship, a second syntax tree structure is constructed for each code change content, and the code submission information is marked in the second syntax tree structure; The second syntax tree structure is matched with multiple syntax subtrees in the first syntax tree structure for similarity. Based on the matching results, the second syntax tree structure is merged into the first syntax tree structure for code tracing.

2. The method according to claim 1, characterized in that, The step of performing similarity matching between the second syntax tree structure and multiple syntax subtrees in the first syntax tree structure includes: A first feature vector is generated based on the node type frequency of each subtree in the first syntax tree structure, and a second feature vector is generated based on the node type frequency of the second syntax tree structure. A hash table is constructed based on the first feature vector, a hash signature is determined based on the second feature vector, and a selection set of candidate syntax subtrees corresponding to the first syntax tree structure is determined based on the hash table and the hash signature. The cosine similarity between the second feature vector and the first feature vector corresponding to each grammar subtree in the candidate grammar subtree selection set is determined, and the cosine similarity is compared with a set threshold to complete the similarity matching process.

3. The method according to claim 2, characterized in that, The step of merging the second syntax tree structure into the first syntax tree structure based on the matching result includes: When the cosine similarity is greater than the set threshold, the corresponding subtree in the first syntax tree structure is replaced by the second syntax tree structure. When the cosine similarity is less than the set threshold, the second syntax tree structure is inserted into the first syntax tree structure as a new syntax subtree.

4. The method according to claim 1, characterized in that, The step of annotating the code commit information in the second syntax tree structure includes: Locate the subtree containing the code change in the second syntax tree structure, and mark the code commit information on the root node of the subtree.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the problem code and convert it into a syntax tree structure to be matched; Determine the first syntax tree structure of the source file corresponding to the problematic code; Based on the syntax tree structure to be matched and the first syntax tree structure, locate the code submission information corresponding to the problematic code.

6. The method according to claim 5, characterized in that, The step of locating the code submission information corresponding to the problematic code based on the syntax tree structure to be matched and the first syntax tree structure includes: Determine the similarity between each subtree in the first syntax tree structure and the syntax tree structure to be matched, and determine the target similar syntax subtree based on the similarity. Obtain the code submission information carried by the target similar syntax subtree to locate the code submission information corresponding to the problematic code.

7. The method according to claim 6, characterized in that, The code submission information includes submitting user identification information, and the method further includes: Retrieve the historical code submission records of the submitting user based on the submitting user identification information; Determine the similarity between the historical code submission records and the problematic code; A warning message is sent to the submitting user based on the similarity and a preset threshold.

8. A code tracing system, characterized in that, The system includes: Data acquisition module, feature extraction module, and matching update module; The data acquisition module is used to acquire source code files and code submission records, wherein the code submission records include code submission information and code change content, and the code change content is associated with the source code files; The feature extraction module is used to construct a corresponding first syntax tree structure for each source code file, wherein the first syntax tree structure includes multiple syntax subtrees; The feature extraction module is further configured to split the code change content according to the association relationship, and construct a second syntax tree structure for each code change content, wherein the second syntax tree structure carries code submission information; The matching update module is used to perform similarity matching between the second syntax tree structure and multiple syntax subtrees in the first syntax tree structure, and merge the second syntax tree structure into the first syntax tree structure according to the matching results.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; and A memory storing computer-readable instructions, which, when executed, cause the processor to perform the operations of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images