ANTLR-based code block identification and dependency tree construction method and device

Abstract

The invention discloses a code block identification and dependency tree construction method and device based on ANTLR. The method comprises the following steps: defining an ANTLR grammar rule file for a target programming language; automatically generating a lexical analyzer and a syntactic analyzer by using an ANTLR tool, and converting the source code into a syntactic analysis tree; traversing the syntactic analysis tree in a monitor mode, introducing a stack data structure to dynamically track the context of a code block, and constructing code block nodes and establishing a father-child hierarchical relationship between the nodes by rewriting an entry method and an exit method related to the code block; constructing a dependency tree based on a hierarchical relationship, wherein each node comprises type, name, position and father-son reference information; and finally rendering and outputting a tree structure. According to the method, the parser is automatically generated through the formalized grammar rule, manual word segmentation and hard coding logic of a traditional method are avoided, and the accuracy, efficiency and maintainability of code block recognition and dependency relationship construction are remarkably improved.

Description

Technical Field

[0001] This application relates to the fields of computer software engineering, program analysis and processing technology, and in particular to a method, apparatus, computer-readable storage medium and electronic device for code block identification and dependency tree construction based on ANTLR (Another Tool for Language Recognition), which is mainly applied to scenarios such as source code parsing, code structure analysis, dependency extraction and software architecture visualization. Background Technology

[0002] In the field of program analysis and understanding, code block identification and dependency tree construction are two fundamental and critical tasks. Code block identification aims to decompose source code into logically independent semantic units (such as functions, classes, control structures, etc.), while dependency tree construction is used to reveal the calls, inclusions, and dependencies between these units. The accuracy and efficiency of these two tasks directly affect the effectiveness of higher-level applications such as code understanding, static analysis, refactoring, debugging, and system architecture recovery.

[0003] 1. Limitations of traditional code block recognition methods

[0004] Early code analysis tools generally employed custom lexical analysis methods implemented in native programming languages, relying on heuristic rules such as keyword matching, bracket counting, and indentation analysis to identify code blocks. This method has the following inherent drawbacks:

[0005] Insufficient accuracy: Modern programming languages ​​have high syntactic complexity, and relying solely on keyword and bracket matching is insufficient to correctly handle all syntactic scenarios. For example, nested subquery structures in SQL often lead to misjudgments of hierarchy boundaries.

[0006] Poor maintainability: The syntax rules of different programming languages ​​differ significantly, requiring separate implementation and maintenance of word segmentation logic for each language; and when the language standard is updated, the corresponding rules must also be modified synchronously, resulting in high maintenance costs.

[0007] Performance bottleneck: To handle complex grammatical scenarios, the word segmentation logic usually contains a large number of conditional branch judgments, which significantly reduces execution efficiency when parsing large-scale code files.

[0008] 2. Limitations of traditional dependency tree construction methods

[0009] Traditional methods typically use hard-coded logic (such as deeply nested if-else or switch-case statements) to infer dependencies between code blocks, for example, identifying stored procedure calls through pattern matching of CALL statements. This method has the following problems:

[0010] Poor flexibility: Hard-coded logic is sensitive to changes in coding style and has difficulty adapting to the diversity of syntactic expressions.

[0011] Poor scalability: Adding new language support or grammatical structures requires modifying the core processing logic, limiting scalability.

[0012] Low reliability: Complex conditional branches are prone to missing boundary conditions, and are prone to misjudgment in scenarios such as dynamic SQL or conditional compilation.

[0013] 3. Other technical challenges

[0014] In addition to the methodological shortcomings mentioned above, traditional implementations also face the dual challenges of performance and accuracy:

[0015] Performance bottleneck: When parsing a code file with tens of thousands of lines, memory consumption may exceed 6GB, causing system sluggishness or even crashes.

[0016] Accuracy limitations: Traditional methods struggle to effectively handle context-dependent grammars, and such structures are prevalent in languages ​​like SQL, making them prone to parsing errors.

[0017] ANTLR technology offers a potential solution to the aforementioned problems by automatically generating lexical analyzers and parsers and constructing a complete parsing tree. However, how to efficiently utilize ANTLR to achieve accurate code block identification and construct a tree structure that accurately reflects the dependencies between code blocks remains a pressing technical challenge. Summary of the Invention

[0018] To address the aforementioned technical requirements and overcome a series of shortcomings in existing technologies, this application proposes a novel ANTLR-based method for code block identification and dependency tree construction.

[0019] In modern software engineering, the analysis and understanding of program code is a fundamental and crucial step. Traditional code analysis methods typically employ manually written parsers or text processing based on regular expressions. These methods are not only inefficient but also struggle to accurately capture the complex syntactic structures of programming languages. To address these issues, this invention introduces ANTLR, an advanced parser generation tool. By defining formal grammar rules, it automatically generates a lexer and a parser, thereby efficiently and accurately converting source code into a structured data model (such as an abstract syntax tree) and further constructing a dependency tree between code blocks.

[0020] This invention is particularly suitable for parsing and processing DDL (Data Definition Language) statements of database programming objects such as database stored procedures, functions, packages, and package bodies. It can also be extended to code analysis of general programming languages ​​such as Java and Python. It can be widely used in software engineering scenarios such as code migration, system refactoring, architecture analysis, and quality assessment, and effectively solves the problems of low accuracy, difficult maintenance, and poor performance in traditional code analysis techniques.

[0021] To achieve the above objectives, the present invention adopts the following technical strategy: the structure of the target programming language (such as SQL, Java, etc.) is described by defining ANTLR syntax rules, and the corresponding lexical analyzer and parser are automatically generated by ANTLR to convert the source code into a syntax analysis tree; then, the syntax analysis tree is traversed by combining the listener pattern with the stack data structure to identify code blocks and establish the hierarchical relationship between them; finally, a clear dependency tree is constructed by rendering the parent-child relationship.

[0022] Specifically, this application provides the following technical solutions:

[0023] The first aspect of this application provides a method for code block identification and dependency tree construction based on ANTLR, the method comprising:

[0024] S1. Define an ANTLR syntax rule file for the target programming language. The ANTLR syntax rule file contains lexical rules describing the basic vocabulary units of the language and syntax rules describing the grammatical structure of the language.

[0025] S2. Based on the ANTLR syntax rule file, automatically generate the corresponding lexical analyzer, parser, and basic listener classes using the ANTLR tool;

[0026] S3. Input the source code of the target programming language into the lexical analyzer to convert it into a lexical symbol stream, and then the syntax analyzer parses the lexical symbol stream into a syntax parse tree according to the syntax rules;

[0027] S4. The parsing tree is traversed using the listener pattern. During the traversal, a stack data structure is introduced to dynamically track the current code block context. By overriding the entry and exit methods related to the code block in the basic listener class, code block nodes are constructed and parent-child hierarchical relationships between code block nodes are established.

[0028] S5. Construct and render the output dependency tree based on the established code block hierarchy. Each code block node in the dependency tree contains at least the code block type, name, source code location information, parent node reference, and child node list.

[0029] Furthermore, in the method of this application, the target programming language mentioned in step S1 includes SQL, Java, and Python;

[0030] The ANTLR syntax rule file uses the .g4 file format and supports the LL(*) parsing algorithm to handle left recursion and syntactic ambiguity.

[0031] Furthermore, in the method of this application, step S3 also includes: using the error recovery mechanism of ANTLR to construct a usable parsing tree in the case of syntax errors in the source code.

[0032] Furthermore, in the method of this application, step S4 specifically includes:

[0033] In the entry method of the overridden base listener class, create the corresponding code block node, establish its association with the parent node at the top of the stack, and push the node onto the stack.

[0034] In the exit method of the overridden base listener class, a stack pop operation is performed to complete the construction of the current code block node.

[0035] Furthermore, in the method of this application, step S4 specifically includes:

[0036] (1) When entering a code block, create a code block node in the corresponding entry method, push it onto the stack, and record its hierarchical relationship with the parent code block represented by the next node in the stack;

[0037] (2) When traversing the statements in the code block, take the top node of the stack as the current code block and record the found sub-code blocks as the child nodes of the current code block node;

[0038] (3) When exiting the code block, pop the code block node from the stack in the corresponding exit method.

[0039] Furthermore, in the method of this application, the root node of the dependency tree in step S5 represents the entire source code script, its child nodes correspond to the top-level code block, and the internal nested code blocks are recursively constructed as the lower-level child nodes of the corresponding parent node.

[0040] Each code block node in the dependency tree also contains symbol table information, data flow information, and code metrics.

[0041] Furthermore, in the method of this application, the rendering output in step S5 includes serializing the dependency tree into JSON or XML format or displaying it as a graphical tree structure.

[0042] Furthermore, the method of this application also includes: when the parsed object is a large-scale source code file, an incremental parsing strategy is adopted, and only key metadata information is saved in the code block node, while the detailed content of the source code is read from the original input as needed, so as to reduce memory usage.

[0043] A second aspect of this application provides an ANTLR-based code block identification and dependency tree construction apparatus, wherein the apparatus, when running, implements the steps of the aforementioned ANTLR-based code block identification and dependency tree construction method, and the apparatus includes:

[0044] The syntax rule definition module is used to define ANTLR syntax rule files for the target programming language. The ANTLR syntax rule files contain lexical rules describing the basic lexical units of the language and syntax rules describing the grammatical structure of the language.

[0045] The code generation module is used to automatically generate corresponding lexical analyzers, parsers, and basic listener classes based on the ANTLR syntax rule file using the ANTLR tool.

[0046] The syntax parsing module is used to input the source code of the target programming language into the lexical analyzer and convert it into a lexical symbol stream, and then the syntax analyzer parses the lexical symbol stream into a syntax parse tree according to the syntax rules;

[0047] The traversal processing module is used to traverse the syntax analysis tree using the listener pattern. During the traversal, a stack data structure is introduced to dynamically track the current code block context. By overriding the entry and exit methods related to the code block in the basic listener class, each code block is identified and the parent-child hierarchical relationship between each code block node is established.

[0048] The dependency tree building module is used to build and render the output dependency tree based on the established code block hierarchy. Each code block node in the dependency tree contains at least the code block type, name, source code location information, parent node reference, and child node list.

[0049] A third aspect of this application provides an electronic device, including: a memory and a processor;

[0050] Memory: Used to store computer programs;

[0051] Processor: Used to execute the computer program to implement the steps of the aforementioned ANTLR-based code block identification and dependency tree construction method.

[0052] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the aforementioned ANTLR-based code block identification and dependency tree construction method.

[0053] In summary, this invention provides a method for code block identification and dependency tree construction based on ANTLR. It defines the language structure through ANTLR syntax rules, utilizes a Listener+Stack mechanism to identify code block hierarchical relationships, and ultimately constructs a complete code dependency tree. Compared to traditional methods, this invention significantly improves accuracy, efficiency, maintainability, and scalability, possessing significant practical value and promising prospects for widespread application. The specific technical advantages of this invention include:

[0054] (1) High accuracy: The ANTLR-based parser strictly follows the language syntax rules and can accurately identify code block boundaries, avoiding errors caused by ambiguity in traditional methods.

[0055] (2) Strong maintainability: The syntax rules and processing logic are separated. When the language syntax changes, only the syntax rule file needs to be adjusted, without modifying the core processing code.

[0056] (3) Good scalability: By defining different syntax rules, the same framework can support the analysis of multiple programming languages, which greatly reduces the cost of expansion.

[0057] (4) High efficiency: Combining the traversal methods of Listener and Stack, the algorithm's time complexity is linearly related to the code size, and it can maintain good performance even when processing large-scale code files.

[0058] (5) Strong fault tolerance: ANTLR’s built-in error recovery mechanism enables the present invention to handle code that does not conform to strict syntax rules to a certain extent (such as code in legacy systems).

[0059] The following table compares the features of this invention with traditional code analysis techniques:

[0060] Table 1: Comparison of the Invention Technology with Traditional Code Analysis Techniques

[0061]

[0062] Other features and advantages of this application will be described in detail in the following description, or can be learned by implementing the relevant technical solutions of this application. Attached Figure Description

[0063] To more clearly illustrate the technical solution of this application, the accompanying drawings involved in the description of this invention will be briefly introduced below. It should be noted that the drawings only show some embodiments of the invention. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.

[0064] Figure 1This is a flowchart illustrating the overall implementation of the ANTLR-based code block identification and dependency tree construction method in this application.

[0065] Figure 2 This is a structural diagram of the code block recognition and dependency tree construction device based on ANTLR in this application.

[0066] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0068] In this document, the term "comprising" and any variations thereof (such as "including," "including," etc.) are open-ended expressions and should be understood as "including but not limited to," meaning that the listed content is not exhaustive and may include other content not explicitly mentioned. The term "based on" should be understood as "at least partially based on," meaning that the basis or condition referred to may not be the only factor and may involve other relevant factors. The term "one embodiment" should be understood as "at least one embodiment," meaning that the described embodiment is not the only possible implementation, and other similar embodiments may exist.

[0069] In this application, the terms "a" and "a plurality of" are used to modify related elements or features, and their expression is illustrative rather than restrictive. Unless otherwise expressly stated in the context, "a" should be understood as "at least one," and "a plurality of" should be understood as "at least two." Those skilled in the art should reasonably interpret these terms based on the semantic and logical relationships of the context to ensure that they cover the possibility of "one or more."

[0070] Example: A method for code block identification and dependency tree construction based on ANTLR

[0071] Figure 1 The diagram shows the overall implementation flow of the ANTLR-based code block identification and dependency tree construction method provided in this application, including the following steps:

[0072] S1. Define an ANTLR syntax rule file for the target programming language. The ANTLR syntax rule file contains lexical rules describing the basic vocabulary units of the language and syntax rules describing the grammatical structure of the language.

[0073] S2. Based on the ANTLR syntax rule file, automatically generate the corresponding lexical analyzer, parser, and basic listener classes using the ANTLR tool;

[0074] S3. Input the source code of the target programming language into the lexical analyzer to convert it into a lexical symbol stream, and then the syntax analyzer parses the lexical symbol stream into a syntax parse tree according to the syntax rules;

[0075] S4. The parsing tree is traversed using the listener pattern. During the traversal, a stack data structure is introduced to dynamically track the current code block context. By overriding the entry and exit methods related to the code block in the basic listener class, code block nodes are constructed and parent-child hierarchical relationships between code block nodes are established.

[0076] S5. Construct and render the output dependency tree based on the established code block hierarchy. Each code block node in the dependency tree contains at least the code block type, name, source code location information, parent node reference, and child node list.

[0077] The technical solution of the present invention will be described in detail below:

[0078] 1. ANTLR-based syntax rule definition

[0079] This invention first defines an ANTLR syntax rule file (usually with a .g4 extension) for the target language. This file contains lexical rules and syntax rules, defining the language's basic vocabulary (such as keywords, identifiers, operators, etc.) and grammatical structures (such as statements, expressions, code blocks, etc.), respectively. Taking SQL stored procedures as an example, the syntax rules need to define elements such as CREATEPROCEDURE, BEGIN-END blocks, and control structures (such as IF-THEN-ELSE, LOOP).

[0080] ANTLR 4 supports the LL() parsing algorithm, enabling it to handle left recursion and complex syntactic ambiguities—features difficult to achieve with traditional parser generation tools. Through precise grammar rule definitions, the parser generated by ANTLR can accurately parse source code into a parse tree*, where each node represents a grammatical structure within the source code. The following example shows a simple SQL procedure statement rule definition:

[0081] procedureDefinition

[0082] : CREATE (OR REPLACE)? PROCEDURE procedureName

[0083] ('(' parameter (',' parameter)* ')')?

[0084] (IS | AS)

[0085] (DECLARE declarations)?

[0086] BEGIN

[0087] statements

[0088] END procedureName? ';' ;

[0090] 2. Code generation and syntax analysis

[0091] After defining the grammar rules, the ANTLR tool is used to generate the corresponding Java code (i.e., the lexical analyzer, parser, and related base classes). These generated classes can convert the input source code into a parse tree.

[0092] Specifically, the source code is input as a string. First, the lexical analyzer converts it into a stream of lexical symbols. Then, the parser organizes this stream into a tree structure according to grammar rules. This process utilizes ANTLR's error recovery mechanism, allowing a usable parser tree to be built even if the source code contains a few syntax errors. This is particularly useful for analyzing legacy systems or projects with non-standard code.

[0093] 3. Code block hierarchy recognition using the combination of Listener and Stack

[0094] ANTLR provides two mechanisms for traversing the parse tree: the listener pattern and the accessor pattern. This invention preferably adopts the listener pattern because it can decouple the traversal and processing logic, making the code clearer and more modular.

[0095] In the listener pattern, ANTLR automatically generates a base listener containing entry and exit methods for each syntax rule. This invention inherits from this base listener and overrides the relevant methods to execute custom logic while traversing different nodes of the tree.

[0096] To accurately construct the hierarchical relationship of code blocks, this invention introduces a stack data structure to track the current code block. The specific process is as follows:

[0097] (1) When entering a code block (such as the body of a stored procedure), a new code block node is created in the enterProcedureDefinition method and pushed onto the stack, while recording its relationship with the parent code block (the next node in the stack).

[0098] (2) When traversing the statements in the code block, the current code block node is always at the top of the stack, and all sub-code blocks (such as nested blocks or control structures) found in the block will be recorded as the child nodes of the current node.

[0099] (3) When exiting the code block, pop the code block node from the stack in the exitProcedureDefinition method to complete the construction of the code block.

[0100] This Listener+Stack mechanism ensures accurate identification of code block levels, and can handle complex nested structures correctly.

[0101] 4. Dependency Tree Construction and Rendering

[0102] After identifying all code blocks and establishing their hierarchical relationships, this invention renders and constructs a complete dependency tree using parent-child relationships. Each code block node contains the following information:

[0103] Type (such as stored procedure, function, package, control block, etc.)

[0104] Name (e.g., stored procedure name)

[0105] Starting position (line number, column number in the source code)

[0106] Parent node reference

[0107] List of child nodes

[0108] The root node of the dependency tree represents the entire DDL script. Its child nodes may be top-level stored procedures or functions, and the child nodes of these nodes may be nested code blocks (such as loops, conditional statements, etc.). This tree structure clearly shows the inclusion relationship and call chain of code blocks, providing a solid foundation for subsequent code analysis, visualization, and optimization.

[0109] To more clearly illustrate the technical solution of this invention, the following detailed description will be provided in conjunction with specific scenario embodiments. This embodiment takes parsing the DDL statements of an Oracle database stored procedure as an example, but it should be understood that this invention is not limited to this, but can be applied to code analysis of various programming languages.

[0110] 1. Environment Preparation and Input Processing

[0111] First, you need to include the ANTLR 4 runtime library in your project. For Maven projects, add the following dependency to pom.xml:

[0112] <dependency>

[0113] <groupid> org.antlr< / groupid>

[0114] <artifactid> antlr4-runtime< / artifactid>

[0115] <version> 4.9.2< / version>

[0116] < / dependency>

[0117] After defining the SQL syntax rules file (e.g., PlSql.g4), use the ANTLR tool to generate Java code:

[0118] antlr4 PlSql.g4

[0119] Next, the DDL statements of the stored procedure / function / package / package body to be analyzed are taken as string input to build the ANTLR parser:

[0120] String ddlCode = "CREATE OR REPLACE PROCEDURE example_procedure ISBEGIN ... END;";

[0121] CharStream input = CharStreams.fromString(ddlCode);PlSqlLexer lexer =new PlSqlLexer(input);

[0122] CommonTokenStream tokens = new CommonTokenStream(lexer);

[0123] PlSqlParser parser = new PlSqlParser(tokens);

[0124] Here, CommonTokenStream serves as a lexical token stream, containing all lexical units extracted from the source code and their location information, providing a foundation for subsequent syntax analysis.

[0125] 2. Syntax Analysis and Tree Traversal

[0126] After obtaining the parser, parsing begins from the entry rule (for DDL scripts, the entry rule is usually `compilation_unit` or `sql_script`), generating a syntax parse tree:

[0127] ParseTree tree = parser.compilation_unit();

[0128] Next, a custom listener is created and the parsing tree is traversed. The custom listener is implemented by inheriting from the base listener class automatically generated by ANTLR, and the entry / exit methods related to the code block are overridden:

[0129] public class CustomBlockListener extends PlSqlParserBaseListener {

[0130] private Stack <codeblock>blockStack = new Stack<>();

[0131] private CodeBlock rootBlock = new CodeBlock("ROOT");

[0132] private List <codeblock>allBlocks = new ArrayList<>();

[0133] public CustomBlockListener() {

[0134] blockStack.push(rootBlock);

[0135] }

[0136] @Override

[0137] public void enterProcedureDefinition(PlSqlParser.ProcedureDefinitionContext ctx) {

[0138] String procedureName = ctx.procedureName().getText();

[0139] CodeBlock procedureBlock = new CodeBlock("PROCEDURE", procedureName);

[0140] / / Establish the relationship with the parent code block

[0141] CodeBlock parent = blockStack.peek();

[0142] parent.addChild(procedureBlock);

[0143] blockStack.push(procedureBlock);

[0144] allBlocks.add(procedureBlock);

[0145] }

[0146] @Override

[0147] public void exitProcedureDefinition(PlSqlParser.ProcedureDefinitionContext ctx) {

[0148] blockStack.pop();

[0149] }

[0150] }

[0151] 3. Dependency tree construction and output

[0152] After traversal is complete, the rootBlock contains the complete code block hierarchy. At this point, the dependency tree can be rendered into the desired format (such as JSON, XML, or a graphical tree structure) through recursive traversal:

[0153] public void renderDependencyTree(CodeBlock block, int level) {

[0154] / / Recursively render child nodes

[0155] for (CodeBlock child : block.getChildren()) {

[0156] renderDependencyTree(child, level + 1);

[0157] }

[0158] }

[0159] 4. Advanced processing and optimization

[0160] For large-scale code files, this invention can employ incremental parsing and memory optimization strategies. For example, in the listener, only key information (such as the type, name, location, and parent-child relationship of code blocks) can be stored in memory, while the detailed content of the code can be read from the original input as needed.

[0161] Furthermore, to enhance the practicality of the dependency tree, this invention can also attach symbol tables, data flow information, or metrics (such as lines of code, complexity, etc.) to code block nodes to support more in-depth code analysis.

[0162] Figure 2 The image shows a code block identification and dependency tree construction device based on ANTLR proposed in this application. The device includes:

[0163] The syntax rule definition module is used to define ANTLR syntax rule files for the target programming language. The ANTLR syntax rule files contain lexical rules describing the basic lexical units of the language and syntax rules describing the grammatical structure of the language.

[0164] The code generation module is used to automatically generate corresponding lexical analyzers, parsers, and basic listener classes based on the ANTLR syntax rule file using the ANTLR tool.

[0165] The syntax parsing module is used to input the source code of the target programming language into the lexical analyzer and convert it into a lexical symbol stream, and then the syntax analyzer parses the lexical symbol stream into a syntax parse tree according to the syntax rules;

[0166] The traversal processing module is used to traverse the syntax analysis tree using the listener pattern. During the traversal, a stack data structure is introduced to dynamically track the current code block context. By overriding the entry and exit methods related to the code block in the basic listener class, each code block is identified and the parent-child hierarchical relationship between each code block node is established.

[0167] The dependency tree building module is used to build and render the output dependency tree based on the established code block hierarchy. Each code block node in the dependency tree contains at least the code block type, name, source code location information, parent node reference, and child node list.

[0168] When the above-mentioned device is in operation, it implements the steps of the ANTLR-based code block identification and dependency tree construction method disclosed in this application.

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of apparatus, methods, and computer program products according to various embodiments of this application, including architecture, functionality, and operation. In these figures, each block may represent a module, program segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should be noted that each block in the block diagrams and / or flowcharts, and combinations thereof, can be implemented using either a dedicated hardware-based system or a combination of dedicated hardware and computer instructions to achieve the specified function or operation.

[0170] like Figure 3 As shown in the illustration, an embodiment of this application also discloses an electronic device, including: a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the ANTLR-based code block identification and dependency tree construction method described above.

[0171] It is understood that, in addition to memory and a processor, this electronic device may also include input devices (such as a keyboard), output devices (such as a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor through I / O interfaces (i.e., input / output interfaces).

[0172] The operations described in this application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include, but are not limited to, the following types:

[0173] Object-oriented programming languages, such as Java, Smalltalk, C++, etc.

[0174] Conventional procedural programming languages, such as "C" or similar programming languages.

[0175] The execution methods of program code include, but are not limited to:

[0176] It runs entirely on the user's computer;

[0177] Part of it executes on the user's computer, and part of it executes on a remote computer;

[0178] Execute as a standalone software package;

[0179] It is executed entirely on a remote computer or server.

[0180] In scenarios involving remote computers, the remote computer can connect to the user's computer via any type of network, including but not limited to local area networks (LANs) or wide area networks (WANs). Furthermore, the remote computer can also connect to external computers through an internet service provider, for example, by utilizing the internet for connection.

[0181] Furthermore, this application also discloses a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by the processor of an electronic device, enables the electronic device to perform the various steps of the ANTLR-based code block identification and dependency tree construction method disclosed in this application.

[0182] In the context of this application, a computer-readable storage medium refers to a tangible medium capable of storing computer program code and related data. Specific examples include, but are not limited to, the following:

[0183] (1) Portable computer disk: such as floppy disks and other removable magnetic storage media.

[0184] (2) Hard disk: including mechanical hard disks and solid-state hard disks and other fixed storage devices.

[0185] (3) Random Access Memory (RAM): A volatile storage medium used for temporary storage of data and program code.

[0186] (4) Read-only memory (ROM): a non-volatile storage medium used to store fixed programs and data.

[0187] (5) Erasable programmable read-only memory (EPROM) or flash memory: non-volatile storage media that supports multiple erasures and reprogrammings.

[0188] (6) Fiber optic storage devices: storage media based on fiber optic technology.

[0189] (7) Portable compact disc read-only memory (CD-ROM): a read-only medium that stores data in the form of an optical disc.

[0190] (8) Optical storage devices: such as DVDs, Blu-ray discs and other storage media based on optical principles.

[0191] (9) Magnetic storage devices: such as magnetic tapes, disks and other storage media based on magnetic principles.

[0192] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage needs.

[0193] These computer-readable storage media can be used to store the program code and related data described in this application to support program execution and persistent data storage.

[0194] Specifically, according to embodiments of this application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of this application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. This computer program includes program code for executing the ANTLR-based code block identification and dependency tree construction method disclosed in this application. When the computer program is executed by a processing device, it can achieve the functions defined in the embodiments of this application.

[0195] While the foregoing discussion contains several specific implementation details, these details should not be construed as limiting the scope of this application. The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features. Furthermore, this application should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the foregoing disclosed concept.

[0196] Those skilled in the art should also understand that modifications can be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features, without departing from the spirit and scope of the technical solutions of the embodiments of this application. These modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of this application.< / codeblock> < / codeblock>

Claims

1. A method for code block identification and dependency tree construction based on ANTLR, characterized in that, The method includes: S1. Define an ANTLR syntax rule file for the target programming language. The ANTLR syntax rule file contains lexical rules describing the basic vocabulary units of the language and syntax rules describing the grammatical structure of the language. S2. Based on the ANTLR syntax rule file, automatically generate the corresponding lexical analyzer, parser, and basic listener classes using the ANTLR tool; S3. Input the source code of the target programming language into the lexical analyzer to convert it into a lexical symbol stream, and then the syntax analyzer parses the lexical symbol stream into a syntax parse tree according to the syntax rules; S4. The parsing tree is traversed using the listener pattern. During the traversal, a stack data structure is introduced to dynamically track the current code block context. By overriding the entry and exit methods related to the code block in the basic listener class, code block nodes are constructed and parent-child hierarchical relationships between code block nodes are established. S5. Construct and render the output dependency tree based on the established code block hierarchy. Each code block node in the dependency tree contains at least the code block type, name, source code location information, parent node reference, and child node list.

2. The method according to claim 1, characterized in that, The target programming languages ​​mentioned in step S1 include SQL, Java, and Python; The ANTLR syntax rule file uses the .g4 file format and supports the LL(*) parsing algorithm to handle left recursion and syntactic ambiguity.

3. The method according to claim 1, characterized in that, Step S3 also includes: using ANTLR's error recovery mechanism to construct a usable parsing tree in the event of syntax errors in the source code.

4. The method according to claim 1, characterized in that, Step S4 specifically includes: In the entry method of the overridden base listener class, create the corresponding code block node, establish its association with the parent node at the top of the stack, and push the node onto the stack. In the exit method of the overridden base listener class, a stack pop operation is performed to complete the construction of the current code block node.

5. The method according to claim 1, characterized in that, Step S4 specifically includes: (1) When entering a code block, create a code block node in the corresponding entry method, push it onto the stack, and record its hierarchical relationship with the parent code block represented by the next node in the stack; (2) When traversing the statements in the code block, take the top node of the stack as the current code block and record the found sub-code blocks as the child nodes of the current code block node; (3) When exiting the code block, pop the code block node from the stack in the corresponding exit method.

6. The method according to claim 1, characterized in that, In step S5, the root node of the dependency tree represents the entire source code script, its child nodes correspond to the top-level code block, and the internal nested code blocks are recursively constructed as the lower-level child nodes of the corresponding parent node. Each code block node in the dependency tree also contains symbol table information, data flow information, and code metrics.

7. The method according to claim 1, characterized in that, The rendering output in step S5 includes serializing the dependency tree into JSON or XML format or displaying it as a graphical tree structure.

8. The method according to claim 1, characterized in that, The method further includes: when the parsed object is a large-scale source code file, an incremental parsing strategy is adopted, and only key metadata information is saved in the code block node, while the detailed content of the source code is read from the original input as needed, so as to reduce memory usage.

9. A code block recognition and dependency tree construction device based on ANTLR, characterized in that, The device includes: The syntax rule definition module is used to define ANTLR syntax rule files for the target programming language. The ANTLR syntax rule files contain lexical rules describing the basic lexical units of the language and syntax rules describing the grammatical structure of the language. The code generation module is used to automatically generate corresponding lexical analyzers, parsers, and basic listener classes based on the ANTLR syntax rule file using the ANTLR tool. The syntax parsing module is used to input the source code of the target programming language into the lexical analyzer and convert it into a lexical symbol stream, and then the syntax analyzer parses the lexical symbol stream into a syntax parse tree according to the syntax rules; The traversal processing module is used to traverse the syntax analysis tree using the listener pattern. During the traversal, a stack data structure is introduced to dynamically track the current code block context. By overriding the entry and exit methods related to the code block in the basic listener class, each code block is identified and the parent-child hierarchical relationship between each code block node is established. The dependency tree building module is used to build and render the output dependency tree based on the established code block hierarchy. Each code block node in the dependency tree contains at least the code block type, name, source code location information, parent node reference, and child node list.

10. An electronic device, characterized in that, include: Memory and processor; Memory: Used to store computer programs; Processor: for executing the computer program to implement the steps of the ANTLR-based code block identification and dependency tree construction method as described in any one of claims 1-8.

Images