A method for extending a programming language syntax

Abstract

This invention discloses a method for extending the syntax of a programming language. The method includes: receiving source code containing lexical annotations; the lexical annotations are defined by annotation definition statements; performing lexical and syntactic analysis on the source code using a preset programming language to generate an abstract syntax tree (AST); identifying the type of annotation nodes during the compilation preparatory stage; processing the AST based on the type of annotation nodes; and compiling the processed AST into a bytecode file using a bytecode compiler, so that the bytecode file can be run through a virtual machine. Through the technical solution of this invention, annotation definition statements are introduced to define the names and scope of lexical annotations in a standardized manner, integrating lexical annotations into the entire compilation process, achieving effective syntax extension; annotation processing is completed during the compilation stage, and the generated bytecode has no additional overhead at runtime, balancing development efficiency and execution performance.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for extending the syntax of a programming language. Background Technology

[0002] With the rapid development of computer science and software engineering, programming languages ​​have become a core infrastructure in fields such as system development, application design, and data processing. Traditional programming languages ​​typically employ fixed syntax and semantics to support specific problem domains or programming paradigms. However, the increasing complexity and diversity of modern software systems have exposed the limitations of existing programming languages ​​in terms of flexibility, ease of use, and scalability, making it difficult to fully meet developers' urgent needs for domain-specific functionality, high-performance abstractions, and rapid iteration. Therefore, how to introduce flexible and efficient syntax extension mechanisms while maintaining the core stability of a language has become an important research topic in the field of programming language design.

[0003] To address these issues, researchers and developers have proposed various grammar extension methods. For example, macros allow pattern-driven code substitution before compilation, enabling custom structures at the grammar level; metaprogramming empowers programs to generate, analyze, and modify code at compile or runtime; and internal domain-specific languages ​​(DSLs) leverage the host language's syntax to build specialized interfaces that closely resemble domain-specific expressions. These methods have, to some extent, enhanced the expressive power of languages ​​and provided developers with means to customize syntax.

[0004] However, existing technologies still have many shortcomings. Macro expansion is often based on text substitution, which easily leads to syntactic ambiguity and is difficult to integrate with type systems, potentially causing compilation errors that are hard to track. Metaprogramming usually requires in-depth modifications to the compiler's internal implementation or runtime semantics, resulting in a very high development threshold and easily introducing complexity and performance overhead. Embedded DSLs are limited by the expressive power of the host language, have a limited scope of expansion, and have high development and maintenance costs, easily leading to a surge in code complexity as the project scales up. In addition, these methods generally lack systematic support for annotation mechanisms. Annotations are usually only used as static metadata markers and cannot participate deeply in the core compilation stages such as lexical analysis, syntax parsing, and code generation, limiting the depth and flexibility of syntactic expansion. Summary of the Invention

[0005] This invention provides a method for extending the syntax of a programming language. By introducing annotation definition statements, the name and scope of lexical annotations are defined in a standardized manner, integrating lexical annotations into the entire compilation process to achieve effective syntax extension. Annotation processing is completed during the compilation phase, and the generated bytecode has no additional overhead at runtime, balancing development efficiency and execution performance. This effectively solves the problems in the prior art, such as the difficulty in balancing syntax extension with the core syntax, high extension threshold, low annotation participation, and lack of constraints on scope.

[0006] In a first aspect, embodiments of the present invention provide a method for extending the syntax of a programming language, comprising: Receive source code containing lexical annotations; the lexical annotations are defined by annotation definition statements; the annotation definition statements include an annotation name field and an annotation scope field; the annotation name field is used to specify one or more annotation identifiers; the annotation scope field is used to specify the types of syntax elements to which the lexical annotations can be applied; the types of syntax elements include at least one of function, class, code block, or call pattern; Lexical and syntactic analysis are performed on the source code using a preset programming language to generate an abstract syntax tree; the abstract syntax tree contains at least one annotation node corresponding to a lexical annotation; During the compilation preparation phase, the type of the annotation node is identified; Based on the type of the annotation node, the abstract syntax tree is processed, and the processed abstract syntax tree is compiled into a bytecode file by a bytecode compiler so that the bytecode file can be run by a virtual machine.

[0007] Optionally, the annotation definition statement further includes: an annotation callback function field; the annotation callback function field is used to specify the annotation processing logic.

[0008] Optionally, the method further includes: if the lexical annotation is a lexical annotation provided by the language layer, then the abstract syntax tree is processed by the built-in lexical annotation, and the processed abstract syntax tree is compiled into a bytecode file by a bytecode compiler; if the lexical annotation is a business lexical annotation, then the abstract syntax tree is converted into code that conforms to the syntax of a preset programming language by a business lexical compiler, and the code is compiled into a bytecode file by a bytecode compiler.

[0009] Optionally, the method further includes: using a business lexical compiler to transform the annotation nodes and their associated code elements in the abstract syntax tree according to the pre-registered annotation processor, to generate code that conforms to the syntax of the preset programming language; the code is in the form of source code of the preset programming language.

[0010] Optionally, the lexical annotations include: lexical annotations provided by the language layer and business lexical annotations; wherein, the lexical annotations provided by the language layer are built into the language core; the lexical annotations provided by the language layer can be used without importing; the business lexical annotations are provided by a third-party framework through a compiler plugin; the business lexical annotations require the framework dependency to be imported into the project.

[0011] Optionally, the compile-time verification operation includes at least one of the following: annotation signature verification, scope conflict detection, type matching verification, and side effect analysis; wherein, the annotation signature verification is used to verify the correctness of annotation usage; the scope conflict detection is used to detect semantic conflicts when multiple annotations act on the same code element; the type matching verification is used to ensure that the return type of the asynchronous wait annotation matches the type of the receiving variable; and the side effect analysis is used to perform side effect analysis on annotations that affect control flow or data flow.

[0012] Secondly, embodiments of the present invention also provide a system for extending the syntax of a programming language, the system comprising: a definition of the programming language, a compiler, and a virtual machine supporting the execution of the programming language; The definition of the programming language includes: syntax rules and semantic rules for lexical annotations; the syntax rules for lexical annotations are used to define the writing form of the lexical annotations; the semantic rules for lexical annotations are used to define how the lexical annotations are processed during the compilation process. The compiler is used to implement the method for extending the syntax of a programming language as described in any of the embodiments of the present invention; the compiler includes: a lexical analyzer, a syntax analyzer, an annotation node recognition module, an annotation node processing module, and a bytecode compiler; The virtual machine is used to execute the bytecode files generated by the compiler and to provide underlying support for the runtime behavior corresponding to the lexical annotations.

[0013] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the methods for extending programming language syntax as provided in any embodiment of the present invention.

[0014] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for extending programming language syntax as provided in any embodiment of the present invention.

[0015] Fifthly, embodiments of the present invention provide a computer program product, including a computer program that, when executed by a processor, implements the method for extending programming language syntax as provided in any embodiment of the present invention.

[0016] The technical solution of this invention receives source code containing lexical annotations. The lexical annotations are defined using annotation definition statements. Each annotation definition statement includes an annotation name field and an annotation scope field. The annotation name field specifies one or more annotation identifiers. The annotation scope field specifies the types of syntax elements to which the lexical annotation can be applied. The syntax element types include at least one of functions, classes, code blocks, or calling patterns. By introducing standardized annotation definition statements, developers can create new annotations clearly and systematically, and precisely control the scope of annotations, avoiding the problems of confusing annotation usage and semantic conflicts in traditional methods, and providing a unified declaration basis for syntax extension. Lexical and syntactic analysis is performed on the source code using a preset programming language to generate an abstract syntax tree. The abstract syntax tree contains at least one annotation node corresponding to a lexical annotation. Annotations are transformed into structured nodes in the abstract syntax tree, enabling them to participate in the compilation process as first-class citizens, laying the foundation for subsequent deep processing, and realizing the leap of annotations from static metadata to programmable components. During the compilation preparation stage, the type of the annotation node is identified. By identifying the annotation type, annotations from different sources are routed to different processing paths, ensuring the relevance and correctness of the processing logic and avoiding the complexity caused by mixed processing. Based on the type of the annotation node, the abstract syntax tree is processed, and the processed abstract syntax tree is compiled into bytecode files by a bytecode compiler, so that the bytecode files can be run by a virtual machine.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a method for extending programming language syntax provided in Embodiment 1 of the present invention; Figure 2This is a flowchart of a method for extending programming language syntax provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a system for extending programming language syntax provided in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the method for extending programming language syntax according to embodiments of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] Example 1 Figure 1 This is a flowchart illustrating a method for extending programming language syntax according to Embodiment 1 of the present invention. This embodiment is applicable to situations requiring flexible syntax extension in programming languages, support for asynchronous programming, resolution of name conflicts, and implementation of declarative programming. This method can be executed by a system for extending programming language syntax, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S110. Receive source code containing lexical annotations; the lexical annotations are defined by an annotation definition statement; the annotation definition statement includes an annotation name field and an annotation scope field; the annotation name field is used to specify one or more annotation identifiers; the annotation scope field is used to specify the types of syntax elements to which the lexical annotation can be applied; the syntax element types include at least one of function, class, code block, or call pattern.

[0023] In this embodiment of the disclosure, a lexical annotation can refer to a marker that begins with a specific symbol (such as "@"). Lexical annotations can be used to attach additional semantic information to code elements and can participate deeply in the compilation process. For example, lexical annotations can be @peer or @await. Source code can refer to program text written by the developer. Source code can be written in a preset programming language (such as Phoenix). Annotation definition statements can refer to language constructs used to define new lexical annotations. For example, annotations are defined in Phoenix using the note statement, with the following format: In this field, x, y, and z are the annotation name fields, lexical(...) is the annotation scope field, and callback is the optional annotation callback function field. The annotation name field can specify one or more annotation identifiers, separated by commas. The annotation scope field can be specified by using the lexical keyword followed by parentheses, indicating the types of syntax elements to which the annotation can be applied. Syntax element types can include: function (fun), class (class), code block (a special unit of code organization) (neta), call pattern (tennis), etc. The annotation identifier can be the name of the annotation, such as peer or await.

[0024] Specifically, the compiler front end receives source code files containing lexical annotations. During the lexical analysis phase, markers starting with "@" are identified and treated as special lexical units. During the syntax analysis phase, annotation definition statements (such as note statements) are parsed into annotation definition nodes according to the language's syntax rules, and information such as the annotation name and its scope is recorded and stored in the symbol table for use in subsequent stages.

[0025] By introducing annotation definition statements, developers can create new lexical annotations in a standardized way and explicitly specify the types of syntactic elements that the annotations can apply to, thereby avoiding the abuse of annotations and semantic conflicts, and providing a unified foundation for the flexible extension of syntax.

[0026] As an optional implementation of this disclosure, the annotation definition statement further includes: an annotation callback function field; the annotation callback function field is used to specify the annotation processing logic.

[0027] In this embodiment of the disclosure, the annotation callback function field can refer to the optional callback part in the annotation definition statement. The annotation callback function field can be used to associate a piece of custom processing logic. The annotation processing logic can refer to the specific operations that the compiler needs to perform when it encounters the annotation, such as generating code, modifying the abstract syntax tree, or performing compile-time verification.

[0028] Specifically, when parsing annotation definition statements, if a `callback` field exists, the compiler will record a reference to that callback function. When the compiler encounters an element decorated with that annotation in the code, it will call the corresponding callback function to perform custom processing.

[0029] Annotation callback functions allow advanced users to inject flexible processing behavior into custom annotations, further enhancing the extensibility of the annotation mechanism without modifying the compiler's core code.

[0030] S120. Perform lexical and syntactic analysis on the source code using a preset programming language to generate an abstract syntax tree; the abstract syntax tree contains at least one annotation node corresponding to a lexical annotation.

[0031] In this embodiment, the preset programming language is Phoenix. Phoenix is ​​an object-oriented and functional programming language that supports lexical annotations. Lexical analysis refers to the process of converting source code character streams into lexical units (Tokens), including identifying keywords, identifiers, operators, annotation tokens, etc. Syntax analysis refers to the process of combining lexical units into grammatical structures according to the language's grammatical rules, generating an Abstract Syntax Tree (AST). An Abstract Syntax Tree is a tree-like representation of the source code's grammatical structure. Each node in the Abstract Syntax Tree corresponds to a grammatical structure, such as function definitions, variable declarations, annotations, etc. Annotation nodes are nodes in the Abstract Syntax Tree that specifically represent lexical annotations. Annotation nodes record the annotation's name, parameters, and the code elements it modifies.

[0032] Specifically, the compiler scans the source code using a lexical analyzer to generate a sequence of lexical units. The parser then organizes these lexical units into an abstract syntax tree (AST) according to the syntax rules of the Phoenix language. During this process, for each lexical annotation, a corresponding annotation node is created in the AST and associated with the code element node it modifies.

[0033] By generating an abstract syntax tree containing annotation nodes, annotations are transformed from simple text tags into structured information that can be recognized and processed by the compiler, laying the data foundation for subsequent annotation recognition and processing.

[0034] S130. During the compilation preparation stage, identify the type of annotation node.

[0035] In this embodiment, the compilation preparation stage can refer to a preprocessing stage after the syntax analysis and AST generation are completed, but before the formal semantic analysis and code generation. The type of annotation node can refer to whether the annotation belongs to a lexical annotation provided by the language layer or a business lexical annotation (i.e., an annotation provided by a third-party framework). Specifically, the compiler traverses the abstract syntax tree, visiting each annotation node. For each annotation node, it retrieves its annotation name and searches for the definition source of that name in the current compilation context. Specifically: first, it checks if it's a built-in annotation of the language core (such as `@peer`, `@await`). If not, it checks if a handler for that annotation is registered in the frameworks imported by the current project. If still not, it checks if there are any user-defined annotations defined using the `note` statement within the current compilation unit. The annotation node is then labeled with its type based on the search results.

[0036] By identifying annotation types, a basis is provided for subsequent dispatch processing, ensuring that different types of annotations can be processed by the correct processor, thus avoiding confusion and conflicts in processing logic.

[0037] As an optional implementation of this disclosure, the lexical annotations include: lexical annotations provided by the language layer and business lexical annotations; wherein, the lexical annotations provided by the language layer are built into the language core; the lexical annotations provided by the language layer can be used without importing; the business lexical annotations are provided by a third-party framework through a compiler plugin; the business lexical annotations require the framework dependency to be introduced in the project.

[0038] In this embodiment, the lexical annotations provided by the language layer can refer to annotations built into the Phoenix language core, such as @peer, @await, @gqn, @case, etc., which can be used directly. Business lexical annotations can refer to annotations provided by third-party frameworks (such as Web frameworks, ORM frameworks) through compiler plugins, such as @route, @transaction. Built into the language core can mean that the definition and processing logic of the annotations are hard-coded in the compiler as part of the language specification. Third-party frameworks can refer to extension libraries developed based on the Phoenix language, such as app.username.net.http. Compiler plugins can refer to modules that conform to the standard compiler interface. Compiler plugins can be used to process framework-defined annotations. A project can refer to a software project under development. Framework dependencies can refer to declarations of references to specific frameworks in the project configuration, such as import app.username.net.http.AsyncClient.

[0039] Specifically, when identifying annotation node types, the compiler determines the annotation source based on the symbol table lookup results. Language-level annotations are processed directly by the compiler internally. Business lexical annotations require invoking the corresponding framework plugin for processing.

[0040] By distinguishing between language layer annotations and business lexical annotations, the core language and ecosystem extensions are decoupled, which not only ensures the stability and efficient compilation of the core, but also provides standardized extension interfaces for third-party frameworks, thus lowering the threshold for framework development.

[0041] S140. Based on the type of the annotation node, process the abstract syntax tree, and use a bytecode compiler to compile the processed abstract syntax tree into a bytecode file so that the bytecode file can be run through a virtual machine.

[0042] In this embodiment of the disclosure, a bytecode compiler can refer to a tool that converts a processed abstract syntax tree (AST) into target bytecode. The processed AST can refer to an AST that may have been modified or enhanced during the annotation processing stage, such as the expansion of language-level annotations or the new AST corresponding to the code converted from business lexical annotations. A bytecode file can refer to a binary file executable on a virtual machine (such as Rhino), typically with the .rse extension. A bytecode file can be a file containing instructions and operands. A virtual machine can refer to the runtime environment of the Phoenix language, responsible for executing bytecode and providing underlying resource management.

[0043] Specifically, depending on the type of annotation node, different processing paths are adopted. For language-level annotations, the compiler directly modifies the abstract syntax tree internally (e.g., converting the code block annotated with @peer into a concurrent execution unit) to obtain the processed AST. For business lexical annotations, the corresponding business lexical compiler is called to convert the original AST into code conforming to Phoenix syntax, and then this code is re-parsed into a new AST (or directly used as an intermediate representation). Finally, the bytecode compiler compiles the processed AST into a bytecode file, which is loaded and executed by the Rhino virtual machine.

[0044] By deeply integrating annotation processing into the entire compilation process, a true extension of programming syntax is achieved. Annotation processing is completed during the compilation phase, and the generated bytecode incurs no additional overhead at runtime, achieving zero-overhead abstraction and balancing development efficiency and execution performance.

[0045] The technical solution of this invention receives source code containing lexical annotations. The lexical annotations are defined through annotation definition statements. These statements include an annotation name field and an annotation scope field. The annotation name field specifies one or more annotation identifiers. The annotation scope field specifies the types of syntax elements to which the lexical annotation can be applied. Syntax element types include at least one of functions, classes, code blocks, or call patterns. By introducing standardized annotation definition statements, developers can create new annotations clearly and systematically, and precisely control the scope of annotations, avoiding the problems of confusing annotation usage and semantic conflicts in traditional methods, and providing a unified declaration basis for syntax extension. Lexical and syntactic analysis is performed on the source code using a preset programming language to generate an abstract syntax tree. The abstract syntax tree contains at least one annotation node corresponding to a lexical annotation. The annotations are transformed into structured nodes in the abstract syntax tree, enabling them to participate in the compilation process as first-class citizens, laying the foundation for subsequent deep processing, and realizing the leap of annotations from static metadata to programmable components. During the compilation preparation phase, the types of annotation nodes are identified. By identifying the annotation type, annotations from different sources are routed to different processing paths, ensuring the relevance and correctness of the processing logic and avoiding the complexity caused by mixed processing. Based on the annotation node type, the abstract syntax tree is processed, and the processed abstract syntax tree is compiled into bytecode files by a bytecode compiler for execution by a virtual machine.

[0046] As an optional implementation of this disclosure, the compile-time verification operation includes at least one of the following: annotation signature verification, scope conflict detection, type matching verification, and side effect analysis; wherein, annotation signature verification is used to verify the correctness of annotation usage; scope conflict detection is used to detect semantic conflicts when multiple annotations act on the same code element; type matching verification is used to ensure that the return type of the asynchronous wait annotation matches the type of the received variable; and side effect analysis is used to perform side effect analysis on annotations that affect control flow or data flow.

[0047] In this embodiment, compilation can refer to the process of converting source code into executable object code. Verification operations can refer to a series of checks performed on annotations and their usage during compilation to ensure program correctness. Annotation signature verification can refer to checking whether the number and type of parameters of the annotation conform to the definition. Scope conflict detection can refer to detecting whether multiple annotations on the same code element cause contradictions; for example, using @peer and @schedule simultaneously may cause conflicts, and the compiler should provide a warning. Type matching verification can refer to ensuring that the return type of the @await expression is consistent with the type of the receiving variable. Side effect analysis can refer to analyzing whether annotations will change control flow (e.g., @await suspends execution) or data flow (e.g., @save saves state) to ensure predictable program behavior. Correctness of annotation usage can refer to the annotation's position, parameters, and target object conforming to its definition. Semantic conflict can refer to multiple annotations having contradictory logic. An asynchronous wait annotation can be @await, used to wait for asynchronous operations to complete in a non-blocking manner. Return type can refer to the data type returned by the asynchronous operation. Receiver variable type can refer to the type of the variable on the left-hand side of the assignment statement. Matching the return type and the receiving variable type can mean they are the same or have an implicit conversion relationship. Control flow annotations, such as `@await` (suspends the current execution flow) and `@schedule` (delays execution), can affect the program's execution path. Data flow annotations, such as `@save` (saves state), can affect data transfer and persistence. Side effects refer to the additional effects that annotations have on the program's state or execution flow.

[0048] Specifically, during the semantic analysis phase of compilation, the compiler calls the verification module to perform the following operations: Annotation signature verification can be used to check whether the parameters of each annotation are valid based on the signature information in the annotation definition. Scope conflict detection can be used to traverse the set of annotations on code elements, determine whether there are mutually exclusive or contradictory annotation combinations, and output warnings or errors. Type matching verification can be used for @await to extract the return type of the asynchronous call and compare it with the type of the receiving variable; if they are inconsistent, an error is reported. Side effect analysis can be used to analyze the impact of annotations on control flow and data flow. For example, @await causes a function to suspend, and the compiler generates corresponding state-saving code accordingly. Similarly, @save requires the compiler to automatically generate a state-saving stack.

[0049] Compile-time verification ensures the correctness and security of annotation usage, avoids runtime errors caused by annotation misuse, and improves program maintainability. Meanwhile, side-effect analysis provides a basis for generating efficient low-level code.

[0050] Example 2 Figure 2This is a flowchart illustrating a method for extending programming language syntax according to Embodiment 2 of the present invention. This embodiment, based on the above embodiments, provides a detailed description of the branching process based on annotation types. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here. Figure 2 As shown, the method includes: S210. Receive source code containing lexical annotations; the lexical annotations are defined by an annotation definition statement; the annotation definition statement includes an annotation name field and an annotation scope field; the annotation name field is used to specify one or more annotation identifiers; the annotation scope field is used to specify the types of syntax elements to which the lexical annotation can be applied; the syntax element types include at least one of function, class, code block, or call pattern.

[0051] S220. Perform lexical and syntactic analysis on the source code using a preset programming language to generate an abstract syntax tree; the abstract syntax tree contains at least one annotation node corresponding to a lexical annotation.

[0052] S230. During the compilation preparation stage, identify the type of annotation node.

[0053] S240. If the lexical annotation is a lexical annotation provided by the language layer, then the abstract syntax tree is processed by the built-in lexical annotation, and the processed abstract syntax tree is compiled into a bytecode file by the bytecode compiler so that the bytecode file can be run by the virtual machine.

[0054] In this embodiment of the disclosure, built-in processing can refer to the compiler directly modifying or enhancing the abstract syntax tree (AST). For example, for the `@peer` annotation, the compiler converts the code block it annotates into a concurrent execution unit and generates work-stealing scheduling code. For the `@await` annotation, the compiler converts it into an asynchronous non-blocking call and generates callback registration and state recovery code. For the `@gqn` annotation, the compiler determines the fully qualified name based on the annotation parameters during the name resolution phase, eliminating name conflicts. The processed AST can refer to the AST after built-in processing. The processed AST may contain new nodes or modified nodes.

[0055] Specifically, when an annotation node is recognized as a lexical annotation provided by the language layer, the compiler invokes the corresponding internal processing module. For example, upon encountering the `@peer` annotation, the internal processor parses the annotation parameters (such as the node list), traverses the annotated code block, converts each task into an independent lightweight execution unit, and inserts task scheduling code. This ultimately generates the modified AST. This AST is then directly compiled into bytecode files by the bytecode compiler.

[0056] Language-level annotations, processed by the compiler's built-in architecture, enable complex syntax extensions (such as concurrency and asynchronicity) in the most efficient way, while ensuring stability and performance. Since no additional plugins are required, developers can use these features out of the box.

[0057] S250. If the type of lexical annotation is business lexical annotation, the abstract syntax tree is converted into code that conforms to the syntax of the preset programming language by the business lexical compiler, and the code is compiled into bytecode file by the bytecode compiler so that the bytecode file can be run by the virtual machine.

[0058] In this embodiment of the disclosure, the business lexical compiler can refer to a compiler module implemented by a third-party framework that conforms to a standard compiler interface. The business lexical compiler can be used to process framework-defined annotations, such as the @route annotation processor provided by a web framework. Code conforming to the syntax of a preset programming language can mean that the code output by the business lexical compiler is source code that conforms to the syntax rules of the Phoenix language. Code conforming to the syntax of a preset programming language can be recognized by a subsequent bytecode compiler.

[0059] Specifically, when an annotation node is identified as a business lexical annotation, the compiler invokes the business lexical compiler registered by the framework. Based on the annotation semantics, the business lexical compiler transforms the annotation node and its associated code elements in the abstract syntax tree, generating source code that conforms to Phoenix syntax. This source code is then re-entered into the compilation process (or directly used as an intermediate representation) and compiled into bytecode files by the bytecode compiler.

[0060] With a business-oriented lexical compiler, third-party frameworks can declaratively extend language functionality without modifying the compiler core. This pluggable architecture lowers the barrier to entry for framework developers.

[0061] As an optional implementation of this disclosure, converting the abstract syntax tree into code conforming to the syntax of a preset programming language using a business lexical compiler may specifically include: using a business lexical compiler to convert annotation nodes and their associated code elements in the abstract syntax tree according to a pre-registered annotation processor, generating code conforming to the syntax of a preset programming language; the code is in the form of source code in the preset programming language.

[0062] In this embodiment, the pre-registered annotation processor can refer to a processor instance registered with the compiler by a third-party framework during initialization, bound to a specific annotation name. The associated code element can refer to the code element modified by the annotation, such as functions, classes, variables, etc., and the statements and expressions they contain. The source code format can refer to the final generated code being text-based Phoenix language source code. The final generated code is readable and can be processed by a standard compiler.

[0063] Specifically, during loading, the third-party framework calls the registration interface provided by the compiler to bind the annotation processor to the annotation name. When the compiler encounters the annotation, it calls the corresponding processor, which traverses the AST subtree where the annotation node is located. Based on the annotation semantics (such as routing paths and transaction boundaries), it generates a new AST node or directly outputs the Phoenix code string. The generated code serves as an intermediate product and is then processed by the subsequent bytecode compiler.

[0064] The Phoenix source code is generated through AST transformation, ensuring that code generated from third-party annotations can be seamlessly integrated into the existing compilation process. The generated code also boasts good readability, facilitating debugging and maintenance. Furthermore, the pre-registration mechanism allows the framework to dynamically extend annotations without requiring recompiling the language compiler.

[0065] In the technical solution of this invention, if the lexical annotation is a language-layer lexical annotation, the abstract syntax tree is processed by the built-in compiler, and the processed abstract syntax tree is compiled into a bytecode file by a bytecode compiler. The language-layer annotation directly modifies the abstract syntax tree through built-in processing, enabling complex syntax extensions (such as concurrency and asynchronicity) in the most efficient way, ensuring the stability and performance of core functions, and allowing developers to write high-performance code in a declarative manner. If the lexical annotation is a business lexical annotation, the abstract syntax tree is converted into code conforming to the syntax of a preset programming language by a business lexical compiler, and the code is compiled into a bytecode file by a bytecode compiler. The business lexical annotation is processed by a pluggable business lexical compiler, providing a standardized, low-barrier syntax extension path for third-party frameworks, enabling framework developers to integrate domain-specific functions in a declarative manner. Simultaneously, it maintains isolation from the language core, ensuring the stability of the language core.

[0066] The following are embodiments of a system for extending programming language syntax provided by the present invention. This system and the methods for extending programming language syntax in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the system for extending programming language syntax, please refer to the embodiments of the methods for extending programming language syntax described above.

[0067] Example 3 Figure 3 This is a schematic diagram of the structure of a system for extending programming language syntax provided in Embodiment 3 of the present invention. Figure 3 As shown, the system includes: the definition of the programming language, the compiler, and the virtual machine that supports the execution of the programming language.

[0068] The definition of the programming language includes: syntactic rules and semantic rules for lexical annotations; the syntactic rules for lexical annotations define the writing form of lexical annotations; the semantic rules for lexical annotations define how lexical annotations are processed during compilation; the compiler is used to implement any of the methods for extending the syntax of the programming language as described in any of the embodiments of the present invention; the compiler includes: a lexical analyzer, a syntax analyzer, an annotation node recognition module, an annotation node processing module, and a bytecode compiler; the virtual machine is used to execute the bytecode files generated by the compiler and provide underlying support for the runtime behavior corresponding to the lexical annotations.

[0069] In this embodiment, semantic rules can refer to defining how lexical annotations are processed during compilation. For example, the `@peer` annotation represents peer-to-peer concurrency, and the compiler needs to convert it into concurrent execution units. The `@await` annotation represents asynchronous waiting, and the compiler needs to generate callback registration and state recovery code. A lexical analyzer can be used to convert the source code character stream into lexical units and identify annotation markers. A syntax analyzer can be used to combine lexical units into an abstract syntax tree according to syntax rules and create annotation nodes. An annotation node identification module can be used to identify the type of annotation node (language layer or business layer) during the compilation preparation stage. An annotation node processing module can be used to call the corresponding built-in processing or business lexical compiler for processing according to the annotation type. A bytecode compiler can be used to compile the processed abstract syntax tree into bytecode files. A virtual machine can be used to execute the bytecode files generated by the compiler and provide underlying support for the runtime behavior corresponding to lexical annotations. For example, the Rhino virtual machine provides lightweight thread scheduling for concurrent tasks generated by the `@peer` annotation and provides an event loop and state recovery mechanism for the `@await` annotation.

[0070] Specifically, the system defines the syntax and semantics of lexical annotations through the programming language definition, enabling the compiler to correctly identify and process annotations. Based on the annotation definitions, the compiler performs lexical and syntactic analysis on the source code, generating an abstract syntax tree containing annotation nodes. During the compilation preparatory stage, it identifies the annotation type and routes it to either the internal processing or the business lexical compiler based on the type. Finally, the bytecode compiler generates bytecode files, which are then executed by the virtual machine. At runtime, the virtual machine provides low-level support for the semantics of annotations (such as concurrency, asynchronicity, and state management), ensuring that the behavior declared in the annotations is correctly implemented.

[0071] This system constructs a complete, efficient, and scalable programming language infrastructure through a three-layer collaboration of language definition, compiler plug-in architecture, and virtual machine runtime support. Developers can enjoy the powerful annotation capabilities built into the language, as well as leverage domain annotations from third-party frameworks, while maintaining code simplicity and high performance.

[0072] Based on the above technical solution, the annotation definition statement also includes: an annotation callback function field; the annotation callback function field is used to specify the annotation processing logic.

[0073] Based on the above technical solution, the compiler is specifically used for: if the lexical annotation type is a lexical annotation provided by the language layer, then the abstract syntax tree is processed by the built-in compiler, and the processed abstract syntax tree is compiled into a bytecode file by the bytecode compiler; if the lexical annotation type is a business lexical annotation, then the abstract syntax tree is converted into code that conforms to the syntax of the preset programming language by the business lexical compiler, and the code is compiled into a bytecode file by the bytecode compiler.

[0074] Based on the above technical solution, the compiler is specifically used to: transform the annotation nodes and their associated code elements in the abstract syntax tree through the business lexical compiler according to the pre-registered annotation processor, and generate code that conforms to the syntax of the preset programming language; the code is in the form of source code of the preset programming language.

[0075] Based on the above technical solution, lexical annotations include: lexical annotations provided by the language layer and business lexical annotations; among them, the lexical annotations provided by the language layer are built into the language core; the lexical annotations provided by the language layer can be used without importing; the business lexical annotations are provided by third-party frameworks through compiler plugins; the business lexical annotations require the framework dependency to be introduced in the project.

[0076] Based on the above technical solutions, the compile-time verification operation includes at least one of the following: annotation signature verification, scope conflict detection, type matching verification, and side effect analysis; wherein, annotation signature verification is used to verify the correctness of annotation usage; scope conflict detection is used to detect semantic conflicts when multiple annotations are applied to the same code element; type matching verification is used to ensure that the return type of asynchronous wait annotation matches the type of the receiving variable; and side effect analysis is used to perform side effect analysis on annotations that affect control flow or data flow.

[0077] The system for extending programming language syntax provided in the embodiments of the present invention can execute the method for extending programming language syntax provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method for extending programming language syntax.

[0078] It is worth noting that in the above embodiments for extending programming language syntax, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0079] Example 4 Figure 4A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile systems, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing systems. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0080] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0081] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0082] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for extending the syntax of programming languages.

[0083] In some embodiments, the method for extending the syntax of a programming language may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for extending the syntax of a programming language described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the method for extending the syntax of a programming language by any other suitable means (e.g., by means of firmware).

[0084] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input system, and at least one output system, and transferring data and instructions to the storage system, the at least one input system, and the at least one output system.

[0085] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing system, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0086] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, system, or device. A computer-readable storage medium can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0087] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display system (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing system (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of systems can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0088] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0089] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0090] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for extending programming language syntax as provided in any embodiment of this application.

[0091] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). This program product belongs to the same inventive concept as the methods for extending programming language syntax disclosed in the embodiments of this application, and therefore will not be described further here.

[0092] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0093] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for extending a programming language syntax, characterized by, include: Receives source code containing lexical annotations; The lexical annotations are defined through annotation definition statements; the annotation definition statements include an annotation name field and an annotation scope field. The annotation name field is used to specify one or more annotation identifiers; the annotation scope field is used to specify the types of syntax elements to which the lexical annotation can be applied; The syntax element type includes at least one of function, class, code block, or calling pattern; Lexical and syntactic analysis are performed on the source code using a preset programming language to generate an abstract syntax tree; The abstract syntax tree contains at least one annotation node corresponding to a lexical annotation; During the compilation preparation phase, the type of the annotation node is identified; Based on the type of the annotation node, the abstract syntax tree is processed, and the processed abstract syntax tree is compiled into a bytecode file by a bytecode compiler so that the bytecode file can be run by a virtual machine.

2. The method of claim 1, wherein, The annotation definition statement also includes an annotation callback function field; the annotation callback function field is used to specify the annotation processing logic.

3. The method of claim 1, wherein, The process of processing the abstract syntax tree based on the type of the annotation node, and then compiling the processed abstract syntax tree into a bytecode file using a bytecode compiler, includes: If the lexical annotation is a lexical annotation provided by the language layer, then the abstract syntax tree is processed by the built-in lexical annotation, and the processed abstract syntax tree is compiled into a bytecode file by the bytecode compiler; If the lexical annotation is of the business lexical annotation type, the abstract syntax tree is converted into code that conforms to the syntax of a preset programming language by a business lexical compiler, and the code is compiled into a bytecode file by a bytecode compiler.

4. The method of claim 3, wherein, The step of converting the abstract syntax tree into code conforming to the preset programming language syntax using a business lexical compiler includes: The business lexical compiler transforms the annotation nodes and their associated code elements in the abstract syntax tree according to the pre-registered annotation processor, generating code that conforms to the syntax of the preset programming language; the code is in the form of source code of the preset programming language.

5. The method according to claim 1, characterized in that, The lexical annotations include: lexical annotations provided by the language layer and business-specific lexical annotations; The lexical annotations provided by the language layer are built into the language core; the lexical annotations provided by the language layer can be used without importing; the business lexical annotations are provided by third-party frameworks through compiler plugins; and the business lexical annotations require the framework dependency to be included in the project.

6. The method according to claim 1, characterized in that, Compile-time verification operations must include at least one of the following: annotation signature verification, scope conflict detection, type matching verification, and side effect analysis. The annotation signature verification is used to verify the correctness of annotation usage; the scope conflict detection is used to detect semantic conflicts when multiple annotations are applied to the same code element; the type matching verification is used to ensure that the return type of the asynchronous wait annotation matches the type of the received variable; and the side effect analysis is used to perform side effect analysis on annotations that affect control flow or data flow.

7. A system for extending the syntax of a programming language, characterized in that, The system includes: a definition of a programming language, a compiler, and a virtual machine that supports the execution of the programming language; The definition of the programming language includes: syntax rules and semantic rules for lexical annotations; the syntax rules for lexical annotations are used to define the writing form of the lexical annotations; the semantic rules for lexical annotations are used to define how the lexical annotations are processed during the compilation process. The compiler is used to implement the method for extending the syntax of a programming language as described in any one of claims 1-6; the compiler includes: a lexical analyzer, a syntax analyzer, an annotation node recognition module, an annotation node processing module, and a bytecode compiler; The virtual machine is used to execute the bytecode files generated by the compiler and to provide underlying support for the runtime behavior corresponding to the lexical annotations.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for extending the syntax of a programming language as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the method for extending the syntax of a programming language as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for extending the syntax of a programming language as described in any one of claims 1-6.

Images