Multi-language function level code difference detection method and system based on syntax tree

Abstract

The invention discloses a multi-language function level code difference detection method and system based on a syntax tree, and the method comprises the steps: S10, carrying out the recursive scanning of a directory of a specified old version code and a directory of a specified new version code, and screening a target language file; s20, analyzing the source code file by using a syntactic analyzer, extracting function definition nodes, and constructing a syntax tree comprising a plurality of functions; s30, generating a unique function identifier corresponding to each function based on the syntax tree; s40, normalizing the function body code corresponding to each function to obtain a normalized result, and generating a hash value based on the normalized result; and S50, according to a judgment rule, comparing the unique function identifier in the old version code with the unique function identifier in the new version code to obtain a code difference result, and completing function-level code difference detection. According to the method, the function level can be directly positioned, the workload of manual screening is remarkably reduced, and the fine granularity and the precision of detection are improved.

Description

Technical Field

[0001] This invention relates to the field of software engineering, and more particularly to a code version management and quality inspection technology, specifically a method and system for detecting differences in multilingual function-level code based on syntax trees. Background Technology

[0002] Existing code diff comparison tools (such as git diff and Beyond Compare) are primarily based on line-by-line text differences. These tools detect differences by comparing line by line, but they have the following shortcomings: 1. Coarse granularity: It cannot be directly located at the function level, and developers need to search manually.

[0003] 2. Prone to false positives: Changes in the style of comments, blank lines, indentation, or curly braces in the code may also be judged as modifications.

[0004] 3. Lack of fully qualified function name matching: It is difficult to accurately distinguish between functions with the same name and overloaded functions in different directories.

[0005] 4. Insufficient report support: Most tools only display in the command line or interface and do not automatically generate structured reports, lacking archiving and traceability functions.

[0006] 5. Although some static analysis tools (such as SonarQube) can parse syntax trees, they are mainly used for code style and security auditing, and are not designed for "function-level difference detection between new and old versions".

[0007] The statements herein provide only background information in relation to this invention and do not necessarily constitute prior art. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for detecting multilingual function-level code differences based on syntax trees, so as to accurately identify code differences at the function level, reduce false alarms caused by comments, whitespace or indentation, and improve the efficiency and accuracy of code version comparison and auditing.

[0009] To achieve the above objectives, this invention provides a multilingual function-level code difference detection method based on a syntax tree, comprising: S10, recursively scanning the directories of specified old and new version code to filter target language files; S20, parsing the source code files using a syntax analyzer to extract function definition nodes and construct a syntax tree containing multiple functions; S30, generating a unique function identifier for each function based on the syntax tree; S40, normalizing the function body code corresponding to each function to obtain a normalization result, and generating a hash value based on the normalization result; S50, comparing the unique function identifiers in the old and new version code according to a judgment rule to obtain code difference results and complete the function-level code difference detection.

[0010] In one embodiment, the determination rule in step S50 includes: if both the old version code and the new version code include functions with the same function identifier, then the determination is made by comparing the hash values ​​of the functions with the same function identifier; if the hash values ​​are different, then the function is determined to be "modified".

[0011] In one embodiment, the determination rule in step S50 further includes: if the function identifier exists in the new version code but does not exist in the old version code, then the function is determined to be "added"; and if the function identifier exists in the old version code but does not exist in the new version code, then the function is determined to be "deleted".

[0012] In one embodiment, the filtering of target language files in step S10 is based on the file extension of the code file; wherein, the target language files include: C / C++ language files and C# language files.

[0013] In one embodiment, for the C# language file, the namespace, class name, function name, and parameter type list are extracted, and the namespace, class name, function name, and parameter type list are sequentially arranged and combined to form a unique function identifier corresponding to each function.

[0014] In one embodiment, for the C / C++ language file, the namespace, class qualifier, function name, and parameter type list are extracted, and the namespace, class qualifier, function name, and parameter type list are sequentially arranged and combined to form the unique function identifier corresponding to each function.

[0015] In one embodiment, the normalization process for the function body code corresponding to each function in step S40 includes: removing blank lines from the function body code; and / or removing comments from the function body code; and / or uniformly indenting the function body code.

[0016] In one embodiment, the syntax tree-based multilingual function-level code difference detection method further includes: S60, classifying the code difference results into "addition, deletion, and modification" and outputting them to a report file.

[0017] In one embodiment, the report file includes at least the file path, function identifier, and code difference results.

[0018] In one embodiment, the report file is a TXT file or a CSV file.

[0019] In one embodiment, the parser is a Tree-sitter parser.

[0020] This invention also provides a syntax tree-based multilingual function-level code difference detection system, applicable to the syntax tree-based multilingual function-level code difference detection method described above. It includes: a reading and filtering module for recursively scanning the directories of specified old and new version code to filter target language files; an analysis and construction module for parsing source code files, extracting function definition nodes, and constructing a syntax tree containing multiple functions; an identifier generation module for generating a unique function identifier for each function; a processing and calculation module, configured with a hash value calculation algorithm, for normalizing the function body code corresponding to each function to obtain a normalization result, and generating a hash value based on the normalization result; and a judgment module, configured with judgment rules, for comparing the unique function identifiers in the old and new version code to obtain code difference results, generating a report file, and completing the function-level code difference detection.

[0021] In one embodiment, the syntax tree-based multilingual function-level code difference detection system further includes: a display module providing a graphical interface for dragging and dropping to select directories, displaying progress bars, logging logs, and displaying report files.

[0022] Compared with the prior art, the method and system for detecting multilingual function-level code differences based on syntax trees of the present invention have at least the following beneficial effects: 1. By constructing a syntax tree containing multiple functions, it is possible to directly locate the function level rather than the line level, significantly reducing the workload of manual screening and improving the fine granularity of detection. 2. Through function body normalization and hash calculation, irrelevant changes such as comments, blank lines, and indentation are automatically ignored, reducing the false positive rate. 3. At the same time, by combining the namespace, class name or class qualifier, function name, and parameter type list corresponding to each function to form a unique fully qualified name, it is possible to distinguish overloaded functions from functions with the same name in different directories, which can effectively improve detection accuracy.

[0023] 2. The aforementioned judgment rules enable rapid detection of code differences between new and old versions, improving detection efficiency while significantly reducing the workload of manual screening. Secondly, the automated generation of structured report files facilitates archiving, comparison, and auditing, ensuring traceability of detection results. Furthermore, the Tree-sitter-based syntax parsing framework allows for rapid extension to other programming languages, giving this invention a strong advantage in scalability. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the multilingual function-level code difference detection method based on syntax trees according to the present invention. Figure 2 This is a schematic diagram of the module of the multilingual function-level code difference detection system based on syntax tree of the present invention; The diagram numbers are as follows: 101 Reading and filtering module, 102 Analysis and construction module, 103 Identifier generation module, 104 Processing and calculation module, 105 Judgment module, and 106 Display module. Detailed Implementation

[0025] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the multilingual function-level code difference detection method and system based on syntax trees proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of this invention. Please refer to the accompanying drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes to aid those skilled in the art and are not intended to limit the implementation conditions of this invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to the size, without affecting the effects and objectives achieved by this invention, should still fall within the scope of the technical content disclosed in this invention.

[0026] To overcome the shortcomings of existing code difference comparison methods, such as coarse granularity, susceptibility to false positives, lack of fully qualified function name matching, and lack of report support, this invention provides a multilingual function-level code difference detection method based on syntax trees, such as... Figure 1As shown, the method includes: S10, recursively scanning the directories of the specified old version code and the new version code to filter target language files; S20, parsing the source code files using a syntax analyzer, extracting function definition nodes, and constructing a syntax tree containing multiple functions; S30, generating a unique function identifier for each function based on the syntax tree; S40, normalizing the function body code corresponding to each function to obtain a normalization result, and generating a hash value based on the normalization result; S50, comparing the unique function identifiers in the old version code and the new version code according to the judgment rules to obtain code difference results, thus completing function-level code difference detection. This embodiment of the invention generates a syntax tree containing multiple functions by parsing the source code of the new and old versions using a syntax tree, generates a unique function identifier for each function, calculates the hash value corresponding to each function, and determines differences based on the judgment rules and function identifiers. This achieves accurate identification of code differences at the function level, reduces the false positive rate, and thus improves detection accuracy.

[0027] Among them, such as Figure 1 As shown, step S10 specifically includes: S11, inputting the directories of the new version code and the old version code to determine the scope of code difference detection and avoid misjudgments caused by irrelevant code for difference detection; S12, recursively scanning files and determining whether they are in the target language to filter target language files. The target language files include C / C++ language files and C# language files. In some embodiments, the method for filtering target language files is based on the file extensions; for example, the extension of a C language file's source file is ".c"; the extension of a C language file's header file is ".h"; the extensions of C++ language file's source files include ".cpp", ".cc", ".cxx", and ".c++"; and the extensions of C++ language file's header files include ".hpp", ".hh", ".hxx", and ".h++".

[0028] In some embodiments, the parser used in step S20 is a Tree-sitter parser, and step S20 specifically includes: S21. Obtain and compile the Tree-sitter language library, initialize the Tree-sitter analyzer, and configure the corresponding language support (such as C / C++ or C#) to ensure that the analyzer can correctly process the syntax rules of the target programming language. S22. Based on the target language files selected in step S10, read the contents of the source code files one by one; S23. Use the Tree-sitter analyzer to perform lexical and syntactic analysis on each source code file to generate a complete abstract syntax tree (AST) representing the structure of the entire file; S24. Define a Tree-sitter query pattern to match function definition nodes, search for all nodes that match the function definition pattern in the generated syntax tree, and use the query language to precisely locate the function declaration, parameter list, and function body; S25. Extract key information from the identified function nodes: namespace, class qualifier, function name and parameter type list, etc. S26. Based on the identification of function definition nodes, a syntax tree containing function names, expressions, and control flow details is constructed by traversing and parsing the substructures of each function node. This embodiment of the invention is based on a Tree-sitter syntax parsing framework, which can be quickly extended to other programming languages, offering the advantage of strong scalability.

[0029] Based on the syntax tree containing multiple functions constructed in step S20, a unique function identifier is generated for each function. Specifically, for the C# language file, the namespace, class name, function name, and parameter type list are extracted, and these elements are sequentially arranged and combined to form the unique function identifier for each function. For the C / C++ language file, the namespace, class qualifier, function name, and parameter type list are extracted, and these elements are sequentially arranged and combined to form the unique function identifier for each function. By combining the namespace, class name or class qualifier, function name, and parameter type list corresponding to each function to form a unique fully qualified name for the function, overloaded functions can be distinguished from functions with the same name in different directories, ensuring that there are no function bodies with the same name. This effectively improves detection accuracy and reduces the false positive rate.

[0030] In a specific embodiment, for a function CreateUser in a C# language file, the extracted namespace is Company.Project.DataAccess, the class name is UserRepository, the function name is CreateUser, and the parameter type list is (string, string, UserType). The unique function identifier corresponding to CreateUser formed by sequentially arranging and combining the extracted namespace, class name, function name, and parameter type list is Company.Project.DataAccess.UserRepository.CreateUser(string, string, UserType).

[0031] The standardization process for the function body code corresponding to each function in step S40 includes: removing blank lines from the function body code; and / or removing comments from the function body code; and / or standardizing the indentation of the function body code. This standardization process reduces false alarms caused by comments, blank spaces, or indentation, improving the efficiency and accuracy of code version comparison and auditing. Furthermore, a hash value for each function is generated based on the standardization result. In some embodiments, a hash value for each function can be generated by calling a hash algorithm (e.g., MD5, SHA1, etc.), and a mapping relationship between the new and old versions of the code directory and the hash values ​​of the functions can be established.

[0032] Based on the above mapping relationship, the function bodies in the new and old versions of the code are compared. The determination rules in step S50 include: if both the old and new versions of the code contain functions with the same function identifier, the determination is made by comparing the hash values ​​of the functions with the same function identifier; if the hash values ​​are different, the function is determined to be "modified". Further, the determination rules in step S50 also include: if the function identifier exists in the new version of the code but not in the old version of the code, the function is determined to be "added"; and if the function identifier exists in the old version of the code but not in the new version of the code, the function is determined to be "deleted". These determination rules can quickly detect differences between the new and old versions of the code, improving detection efficiency while significantly reducing the workload of manual screening.

[0033] Furthermore, the multilingual function-level code difference detection method based on syntax trees provided in this embodiment of the invention further includes: S60, classifying the code difference results into "added, deleted, modified" categories, and outputting them to a report file. The report file at least includes a file path, function identifier, and code difference results (i.e., "added, deleted, modified"). In some embodiments, the report file type is a TXT file or a CSV file. This embodiment of the invention automatically generates structured report files, facilitating archiving, comparison, and auditing, and making the detection results traceable.

[0034] Accordingly, this embodiment of the invention also provides a multilingual function-level code difference detection system based on a syntax tree, applicable to the multilingual function-level code difference detection method based on a syntax tree as described in any of the above embodiments. It includes: a reading and filtering module 101, used to recursively scan the directories of specified old and new version code to filter target language files; an analysis and construction module 102, used to parse source code files, extract function definition nodes, and construct a syntax tree containing multiple functions; an identifier generation module 103, used to generate a unique function identifier for each function; a processing and calculation module 104, configured with a hash value calculation algorithm, used to normalize the function body code corresponding to each function to obtain a normalization result, and generate a hash value based on the normalization result; and a judgment module 105, configured with judgment rules, used to compare the unique function identifiers in the old and new version code to obtain code difference results, generate a report file, and complete the function-level code difference detection.

[0035] Furthermore, the syntax tree-based multilingual function-level code difference detection system also includes a display module 106 providing a graphical interface for dragging and dropping to select directories, displaying progress bars, logging logs, and showing report files. The syntax tree-based multilingual function-level code difference detection system of this invention provides a graphical interface and a background thread mechanism, resulting in a user experience superior to tools that only support command lines.

[0036] In summary, the multilingual function-level code difference detection method and system based on syntax trees provided by this invention can directly locate the function level rather than the line level by constructing a syntax tree containing multiple functions, significantly reducing the workload of manual screening and improving the fine granularity of detection. Secondly, through function body normalization and hash calculation, irrelevant changes such as comments, blank lines, and indentation are automatically ignored, reducing the false positive rate. At the same time, by forming a unique fully qualified name for each function by combining the namespace, class name or class qualifier, function name, and parameter type list corresponding to each function, overloaded functions and functions with the same name in different directories can be distinguished, which can effectively improve the detection accuracy.

[0037] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0038] In the description of this invention, it should be understood that the terms "center," "height," "thickness," "upper," "lower," "vertical," "horizontal," "top," "bottom," "inner," "outer," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0039] In the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0040] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0041] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A method for detecting multilingual function-level code differences based on syntax trees, characterized in that, include: S10. Recursively scan the specified directories of old and new version code to filter target language files; S20. Use a syntax analyzer to parse the source code file, extract function definition nodes, and construct a syntax tree containing multiple functions; S30. Based on the syntax tree, generate a unique function identifier for each function; S40. Normalize the function body code corresponding to each function to obtain a normalization result, and generate a hash value based on the normalization result; S50. According to the judgment rules, compare the unique function identifiers in the old version code and the new version code to obtain the code difference results and complete the function-level code difference detection.

2. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 1, characterized in that, The determination rule in step S50 includes: if both the old version code and the new version code include functions with the same function identifier, then the determination is made by comparing the hash values ​​of the functions with the same function identifier; if the hash values ​​are different, then the function is determined to be "modified".

3. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 2, characterized in that, The determination rule described in step S50 also includes: If the function identifier exists in the new version of the code, but does not exist in the old version of the code, then the function is determined to be "new"; and If the function identifier exists in the old version of the code but not in the new version of the code, then the function is determined to be "deleted".

4. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 1, characterized in that, The filtering of target language files in step S10 is based on the file extension of the code file; wherein, the target language files include: C / C++ language files and C# language files.

5. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 4, characterized in that, For the C# language file, the namespace, class name, function name, and parameter type list are extracted, and the namespace, class name, function name, and parameter type list are arranged and combined in sequence to form the unique function identifier corresponding to each function.

6. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 4, characterized in that, For the C / C++ language file, the namespace, class qualifier, function name, and parameter type list are extracted, and the namespace, class qualifier, function name, and parameter type list are arranged and combined in sequence to form the unique function identifier corresponding to each function.

7. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 1, characterized in that, Step S40, which involves standardizing the function body code for each function, includes: removing blank lines from the function body code; and / or removing comments from the function body code; and / or uniformly indenting the function body code.

8. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 3, characterized in that, Also includes: S60. Categorize the code difference results into "addition, deletion, and modification" and output them to a report file.

9. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 8, characterized in that, The report file must contain at least the file path, function identifier, and code difference results.

10. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 9, characterized in that, The report file can be a TXT file or a CSV file.

11. The method for detecting multilingual function-level code differences based on syntax trees as described in claim 1, characterized in that, The parser is a Tree-sitter parser.

12. A syntax tree-based multilingual function-level code difference detection system, applicable to the syntax tree-based multilingual function-level code difference detection method as described in any one of claims 1-11, characterized in that, include: The reading and filtering module is used to recursively scan the directories of the specified old version code and the new version code to filter target language files; The analysis and building module is used to parse source code files, extract function definition nodes, and build a syntax tree containing multiple functions; The identifier generation module is used to generate a unique function identifier for each function. The processing and calculation module is equipped with a hash value calculation algorithm, which is used to normalize the function body code corresponding to each function to obtain a normalized result, and generate a hash value based on the normalized result; The judgment module is configured with judgment rules to compare the unique function identifiers in the old version code and the new version code, obtain the code difference results, generate a report file, and complete the function-level code difference detection.

13. The multilingual function-level code difference detection system based on syntax trees as described in claim 12, characterized in that, Also includes: Provides a graphical interface display module for drag-and-drop directory selection, progress bar prompts, log display, and report file display.

Images