Real-time compiling method and device, equipment and storage medium

Abstract

The invention discloses a real-time compiling method and device, equipment and a storage medium, belongs to the field of data processing, and can solve the problem that a program cannot be compiled on a target platform in real time. The method comprises the following steps: converting source codes in a character stream into a symbol set through a lexical analysis state machine; processing the grammar symbols in the symbol set through an analysis engine to generate corresponding intermediate representations; the intermediate representation is processed through an assembly generator and a node distributor, a binary file corresponding to a target platform is obtained, and the binary file at least comprises a first machine instruction corresponding to the target platform; and compiling the dynamic code through a real-time compiling engine to obtain a second machine instruction corresponding to the target platform, and putting the second machine instruction into a dynamic loading library to load and run the second machine instruction.

Description

Technical Field

[0001] This application belongs to the field of data processing, specifically relating to a real-time compilation method, apparatus, device, and storage medium. Background Technology

[0002] Programming languages ​​generally need to be translated into corresponding machine instructions to run on a specific platform. In existing technologies, for compiled languages, a compiler translates the source code into machine instructions for execution on the corresponding platform; for interpreted languages, the source code is converted into an intermediate form, and then an interpreter assists in execution.

[0003] In existing technologies, when making changes to a program, it is generally necessary to stop the process, recompile the language, and restart the process on the platform, which affects the normal operation of the platform. Summary of the Invention

[0004] The purpose of this application is to provide a real-time compilation method, apparatus, device, and storage medium that can solve the problem of not being able to compile programs on a target platform in real time.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a real-time compilation method, the method comprising: The source code in the character stream is converted into a symbol set by using a lexical analysis state machine; The analysis engine processes the grammar symbols in the symbol set to generate corresponding intermediate representations; The intermediate representation is processed by an assembler and a section allocator to obtain a binary file corresponding to the target platform. The binary file includes at least the first machine instructions corresponding to the target platform. The dynamic code is compiled by a real-time compilation engine to obtain the second machine instructions corresponding to the target platform, and the second machine instructions are put into a dynamic loading library to load and run the second machine instructions.

[0006] Optionally, the step of converting the source code in the character stream into a symbol set through a lexical analysis state machine includes: The lexical analysis state machine reads characters from the character stream in a loop. The state transition is performed based on the current state and the characters, and it is determined whether the sequence of characters conforms to the preset syntax rules. If the sequence of characters conforms to a preset syntax rule, the corresponding grammar symbol for the character is generated; The grammar symbol corresponding to each character is added to the set to obtain the symbol set.

[0007] Optionally, before converting the source code in the character stream into a symbol set via a lexical analysis state machine, the method further includes: The source code is identified to determine whether it meets preset identification conditions; If the source code meets the preset recognition conditions, the source code is saved to the character stream.

[0008] Optionally, the step of processing the symbols in the symbol set through an analysis engine to generate corresponding intermediate representations includes: The grammar symbols in the symbol set are read by the analysis engine; Based on the grammar symbols, a corresponding abstract syntax tree is constructed using a recursive descent method. The abstract syntax tree includes multiple nodes and a corresponding list of basic code blocks. Each node in the abstract syntax tree is transformed to obtain the intermediate representation.

[0009] Optionally, the step of processing the intermediate representation through an assembler and a section allocator to obtain the binary file corresponding to the target platform includes: The intermediate representation is read by the assembler to obtain the corresponding first machine instruction; The section allocator generates the content of the corresponding section in the binary file according to the corresponding definition in the intermediate representation, thereby obtaining the binary file.

[0010] Optionally, the step of compiling the dynamic code using a real-time compilation engine to obtain the second machine instructions corresponding to the target platform includes: In the case that the dynamic code is one-time executable code, the intermediate representation of the dynamic code is compiled through a preset interface to obtain the second machine instruction; When the dynamic code is on-demand executable code, the intermediate representation of the dynamic code is compiled into a preset code module to obtain the second machine instruction.

[0011] Optionally, the method further includes: If an error occurs during compilation, record the location of the erroneous code, the cause of the error, and related variable information.

[0012] Secondly, embodiments of this application provide a real-time compilation apparatus, the apparatus comprising: The symbol set acquisition module is used to convert the source code in the character stream into a symbol set through a lexical analysis state machine; The intermediate representation generation module is used to process the grammar symbols in the symbol set through the analysis engine to generate corresponding intermediate representations; The first machine instruction generation module is used to process the intermediate representation through an assembler and a section allocator to obtain a binary file corresponding to the target platform. The binary file includes at least the first machine instruction corresponding to the target platform. The second machine instruction generation module is used to compile dynamic code through a real-time compilation engine to obtain the second machine instructions corresponding to the target platform, and put the second machine instructions into a dynamic loading library to load and run the second machine instructions.

[0013] Optionally, the symbol set acquisition module includes: A character reading submodule is used to cyclically read characters from the character stream through the lexical analysis state machine; The character conversion submodule is used to perform state conversion based on the current state and the character, and to determine whether the sequence of characters conforms to preset syntax rules; The grammar symbol generation submodule is used to generate the grammar symbol corresponding to the character when the sequence of characters conforms to the preset grammar rules; The symbol set acquisition submodule is used to add the grammar symbol corresponding to each character to a set to obtain the symbol set.

[0014] Optionally, the device further includes: The verification module is used to identify the source code and determine whether the source code meets the preset identification conditions. The code saving module is used to save the source code to the character stream if the source code meets the preset recognition conditions.

[0015] Optionally, the intermediate representation generation module includes: The grammar symbol reading submodule is used to read the grammar symbols in the symbol set through the analysis engine; The abstract syntax tree construction submodule is used to construct the corresponding abstract syntax tree based on the grammar symbols using a recursive descent method. The abstract syntax tree includes multiple nodes and a corresponding list of basic code blocks. The intermediate representation generation submodule is used to transform each node in the abstract syntax tree to obtain the intermediate representation.

[0016] Optionally, the first machine instruction generation module includes: The first machine instruction generation submodule is used to read the intermediate representation through the assembler to obtain the corresponding first machine instruction; The binary file acquisition submodule is used to generate the content of the corresponding section in the binary file according to the corresponding definition in the intermediate representation through the section allocator, thereby obtaining the binary file.

[0017] Optionally, the second machine instruction generation module includes: A one-time code conversion submodule is used to compile an intermediate representation of the dynamic code through a preset interface to obtain the second machine instruction when the dynamic code is one-time executable code; The on-demand code conversion submodule is used to compile the intermediate representation of the dynamic code into a preset code module to obtain the second machine instruction when the dynamic code is on-demand code.

[0018] Optionally, the device further includes: The error logging submodule is used to record the location of the erroneous code, the cause of the error, and related variable information when an error occurs during compilation.

[0019] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0020] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0021] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0022] In the real-time compilation method provided in this application, the source code in the character stream is converted into a symbol set through a lexical analysis state machine; the grammar symbols in the symbol set are processed by an analysis engine to generate a corresponding intermediate representation; the intermediate representation is processed by an assembler and a section allocator to obtain a binary file corresponding to the target platform, the binary file including at least a first machine instruction corresponding to the target platform; the dynamic code is compiled by a real-time compilation engine to obtain a second machine instruction corresponding to the target platform, and the second machine instruction is placed in a dynamic loading library to load and run the second machine instruction.

[0023] In this method, for source code, a symbol set is generated through a lexical analysis state machine, and then an intermediate representation corresponding to the symbol set is obtained through an analysis engine. The corresponding binary file is then generated through an assembler and a section allocator. For dynamic code, it is compiled through a real-time compilation engine, and the compiled instructions are placed into a dynamic loading library for execution. Dynamic code changes can be achieved without stopping the process and replacing the binary file, saving manpower and resources. Attached Figure Description

[0024] Figure 1 This is a flowchart of a real-time compilation method proposed in an embodiment of this application; Figure 2 This is a schematic diagram of a compiler system proposed in one embodiment of this application; Figure 3 This is a flowchart illustrating the lexical analysis state machine proposed in an embodiment of this application; Figure 4 This is a diagram showing the code block jump relationship after the if statement is transformed according to an embodiment of this application; Figure 5 This is a schematic diagram of a real-time compilation apparatus according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] The real-time compilation method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] First, some terms appearing in the embodiments of this application will be explained: LLVM: (This is the full name, not the abbreviation) An open-source compiler infrastructure project that includes a series of modular compiler components and toolchains.

[0029] JIT: Just-In-Time compilation, in the field of compilation, refers to a compilation technique that can dynamically execute code and obtain results. Unlike interpreted languages ​​executed by a virtual machine, JIT generally refers specifically to compiling code directly into machine code for the target platform and then executing it.

[0030] Target platform: In the field of compilation, the target platform refers to the execution environment of the files generated by the compilation results, which generally includes the corresponding CPU instruction set and operating system.

[0031] AST: Abstract Syntax Tree, used in the field of compilation to describe the structure of source code, is generally the product of syntactic and semantic analysis of source code.

[0032] Basic code block: A term in the compiler field. It refers to a collection of expressions executed sequentially without any other jumps within it. A complete piece of code consists of multiple basic code blocks and the jump relationships between them.

[0033] IR: Intermediate Representation, a term in the compiler field, refers to an intermediate state generated after source code processing, which is easily understood by the machine. Examples include text-based assembly language or custom formats. It serves as a bridge between text-based source code and binary machine instructions. Currently, the industry's main implementation of compilation is divided into a compiler front-end and a back-end, with IR generally serving as the output of the front-end and the input of the back-end.

[0034] refer to Figure 1 , Figure 1 This is a flowchart of a real-time compilation method proposed in an embodiment of this application. This method is applied to a compiler, such as... Figure 1 As shown, the method specifically includes the following steps: S11: Convert the source code in the character stream into a symbol set through a lexical analysis state machine.

[0035] In this embodiment, the lexical analysis state machine is a core component in compiler theory used to convert character streams into sequences of grammar tokens. It is implemented based on finite state automata theory. During lexical analysis, the state machine reads characters from the source program one by one, determines the next state based on the current state and the input character, and thus identifies different types of words. The source code is the code that needs to run on the target platform. The token set is the collection of text characters after the content of the source code has been analyzed and converted. A character stream is a data stream used in computers to process text data; it uses characters (such as letters, numbers, symbols, etc.) as the basic units for reading and writing operations.

[0036] In this embodiment, lexical analysis is the process of converting source code in the form of a text string into a symbol set that conforms to the language definition. Through a lexical analysis state machine, characters in the character stream are read, and state transitions are performed based on the current state and the input characters. After determining whether it conforms to preset syntax rules, a grammar symbol carrying positional information (row and column) is generated using the read characters. Besides the character itself, the grammar symbol also has different attributes depending on its type; for example, an int symbol will have a calculated value. The converted symbol is added to the symbol set. After all character streams have been read and processed, the entire code is converted into a symbol set, which serves as the input for the next step of the syntax and semantic analysis system.

[0037] For example, for the expression sum = 3 + 2, lexical analysis will generate symbol units such as <identifier, sum>, <operator, =>, <number, 3>.

[0038] S12: The grammar symbols in the symbol set are processed by the analysis engine to generate the corresponding intermediate representation.

[0039] In this embodiment, the analysis engine constructs a corresponding syntax tree based on grammar symbols, and then generates a corresponding intermediate representation based on the syntax tree. An intermediate representation (IR) refers to an intermediate state generated after source code processing that is easily understood by the machine, such as text-based assembly language or custom formats. It serves as a bridge between text-based source code and binary machine instructions. Currently, the main implementation of compilation in the industry is divided into a compiler front-end and a back-end, with IR generally serving as the output of the front-end and the input of the back-end.

[0040] In this embodiment, the compiler reads grammar symbols from the symbol set through the analysis engine, processes the symbols using recursive descent, and finally generates the IR (Instant Interpretation Tree). In this engine, the abstract syntax tree includes various nodes and corresponding lists of basic code blocks. Nodes describe each object in the production rule, while code blocks describe the specific logic of the statement. The engine also includes a global IR constructor, which maintains an insertion point corresponding to a specific basic code block. This insertion point is responsible for converting the expression into the IR and writing it into the corresponding code block. This insertion point is frequently switched as needed during the analysis process.

[0041] In this embodiment, each basic code block contains a list of expressions to be executed sequentially (such as addition, subtraction, multiplication, and division). To support branching structures such as if and for function calls, each code block also needs an exit jump instruction. In scenarios requiring a jump, the exit pointer is pointed to the corresponding target code block.

[0042] For example, the syntax of the if statement and the conversion relationship of code block jumps can be represented as follows: if (cond) { / / Logic executed when cond is true } else { / / Logic executed when cond is false } / / Logic after if statement ends Without considering internal logic jumps, this statement involves a conditional expression `cond` and four statement blocks: one is the current statement block when the `if` statement is analyzed, called `curBlock`; the other three are `thenBlock`, which executes code when `cond` is true; `elseBlock`, which executes code when `cond` is false; and `tailBlock`, which executes code after the `if` statement ends.

[0043] The exit jump instructions for these statement blocks are as follows: curBlock -> cond conditional jump to thenBlock or elseBlock.

[0044] thenBlock -> jump directly to tailBlock elseBlock -> jump directly to tailBlock When generating IR for the above code, the IR constructor will assign the insertion point to each of the code blocks when it encounters different statements. Once the code block is pointed to, the corresponding expression will be generated in that code block.

[0045] S13: The intermediate representation is processed by an assembler and a section allocator to obtain a binary file corresponding to the target platform. The binary file includes at least the first machine instruction corresponding to the target platform.

[0046] In this embodiment, the assembler generator is used to compile the intermediate representation of the source code into corresponding assembly language instructions. The section allocator is used to create and fill the various "sections" required by the ELF format in the final output object file according to the definitions of data and code in its intermediate representation (IR). ELF (Executable and Linkable Format) files are a standard binary file format on Unix-like systems (such as Linux) and are used for object files (.o files), executable files, shared libraries (.so files), etc. An ELF file mainly consists of four parts: ELF header, program header table, section header table, and actual sections. Among them, the "section" is the basic organizational unit in the ELF file. Each section carries a specific type of information, such as the .text section which stores program instructions, the .rodata section which stores read-only data, and the .data section which stores initialized global variables, etc. The first machine instructions are the machine instructions compiled from the source code.

[0047] In this embodiment, the IR is read line by line by line by the assembler and translated into the corresponding assembly instructions. Then, the corresponding section content in the target file is generated by the section allocator according to the corresponding definition in the IR.

[0048] In this embodiment, the source code is converted into the first machine instructions of the target platform, and an object file is generated. Finally, a binary file is output through linking.

[0049] S14: Compile the dynamic code using a real-time compilation engine to obtain the second machine instructions corresponding to the target platform, and put the second machine instructions into a dynamic loading library to load and run the second machine instructions.

[0050] In this embodiment, dynamic code is code that needs to be added when the already compiled program is run.

[0051] The second set of machine instructions are machine instructions compiled from dynamically generated code. In this embodiment, dynamic code is compiled by a real-time compilation engine. This engine can compile the code that needs to be executed dynamically and generate an instruction reference (IR). The IR is then used to generate instructions for the target platform and loaded as a dynamic library. After that, the program entry point is switched to the pre-entry point of the dynamic library, and this part of the code can be executed and the execution result can be obtained.

[0052] In this embodiment, the corresponding source code, i.e., static code, is compiled into a corresponding binary file and run on the platform. For dynamic code, it is directly compiled into the corresponding second machine instructions and placed into the dynamic loading library for calling. The compilation and loading of dynamic code can be completed without stopping the system, reducing resource consumption and improving the user experience.

[0053] In another embodiment of this application, the step of converting the source code in the character stream into a symbol set through a lexical analysis state machine includes: S21: Read characters from the character stream cyclically using the lexical analysis state machine.

[0054] In this embodiment, characters are first read from the input stream continuously in a loop using a lexical analysis state machine.

[0055] S22: Perform a state transition based on the current state and the character, and determine whether the sequence of the character conforms to the preset syntax rules.

[0056] In this embodiment, the lexical analysis state machine performs state transitions based on the current state and characters. The state machine has multiple pre-set states. When an input character is read, it determines which state the character belongs to, transitions the state based on the read character, records each read character, and simultaneously determines whether the character sequence conforms to the preset syntax rules.

[0057] S23: If the sequence of characters conforms to the preset syntax rules, generate the grammar symbol corresponding to the character.

[0058] In this embodiment, if the sequence of characters conforms to preset grammar rules, the corresponding grammar symbols for the characters are generated.

[0059] For example, suppose the input character stream is: "123+45", Initial state S0, read in '1' (number) -> according to the rules, switch from S0 to S1 and start recording the number "1".

[0060] State S1 (recognize integer), read in '2' (number) -> according to the rules, stay in S1, and record the number as "12".

[0061] Read in '3' (number) -> According to the rules, keep it in S1 and record the number as "123".

[0062] Read in '+' (not a number) -> according to the rules, end the recognition, generate an integer Token (value = 123), and convert it back to S0.

[0063] In state S0, read in '+' (plus sign) -> generate a plus sign Token according to the rules and convert it to S2.

[0064] Status S2 (plus sign detected) will automatically switch back to S0.

[0065] State S0, read in '4' (number) -> switch to S1, start recording the number "4".

[0066] Read in '5' (number) -> Hold in S1, record the number as "45".

[0067] Input ends -> Recognition ends, and an integer Token (value = 45) is generated.

[0068] Through the above process, the state machine successfully converted the character stream "123+45" into three tokens according to the clear conversion rules: integer (123) -> plus sign (+) -> integer (45).

[0069] S24: Add the grammar symbol corresponding to each character to the set to obtain the symbol set.

[0070] In this embodiment, the grammar symbol corresponding to each character is added to the same set to obtain the corresponding symbol set.

[0071] In this embodiment, the symbol stream is analyzed by a lexical analysis state machine to obtain the corresponding grammatical symbols, which form a symbol set and provide a data foundation for subsequent analysis.

[0072] In another embodiment of this application, before converting the source code in the character stream into a symbol set using a lexical analysis state machine, the method further includes: S31: Identify the source code and determine whether the source code meets the preset identification conditions.

[0073] In this embodiment, the source code is first identified to determine whether it meets the preset identification conditions.

[0074] For example, the preset recognition criteria are: checking whether it is a text file, whether it can be recognized by UTF-8 encoding, etc. S32: If the source code meets the preset recognition conditions, save the source code to the character stream.

[0075] In this embodiment, if the source code meets the preset recognition conditions, the source code is saved to a character stream. If the source code file is too large, the character stream will first cache a portion of the characters at the beginning (e.g., a maximum of 65536 characters). After the cached character set is read, the next batch will be read and added to the cache through the file's I / O interface until the entire file is processed.

[0076] In this embodiment, the source code is placed in a character stream so that the source code can be processed by the compiler.

[0077] In another embodiment of this application, the step of processing the symbols in the symbol set through an analysis engine to generate corresponding intermediate representations includes: S41: Read the grammar symbols in the symbol set through the analysis engine.

[0078] In this embodiment, when generating intermediate representations, the grammar symbols in the symbol set are first read by the analysis engine.

[0079] S42: Based on the grammar symbols, construct the corresponding abstract syntax tree using a recursive descent method. The abstract syntax tree includes multiple nodes and a corresponding list of basic code blocks.

[0080] In this embodiment, after obtaining the grammar symbols, the corresponding abstract syntax tree is constructed by recursive descent. The abstract syntax tree includes multiple nodes and a list of basic code blocks corresponding to each node.

[0081] For example, suppose we want to process the expression: a = 3 + 5 * (2 - 1), the symbol set generated is: [IDENTIFIER("a"), ASSIGN, NUMBER(3), PLUS, NUMBER(5), MULTIPLY, LPAREN, NUMBER(2), MINUS,NUMBER(1), RPAREN, EOF] The recursive descent parser processes expressions by priority: it identifies assignment statements like `a = ...`, handles addition expressions like `3 + ...`, processes multiplication expressions like `5 * ...`, and handles parentheses expressions like `(2 - 1)`. The resulting syntax tree structure is as follows: Assignment ├── Identifier("a") └── BinaryExpr(PLUS) ├── Number(3) └── BinaryExpr(MULTIPLY) ├── Number(5) └── BinaryExpr(MINUS) ├── Number(2) └── Number(1) S43: Transform each node in the abstract syntax tree to obtain the intermediate representation.

[0082] In this embodiment, after generating the abstract syntax tree, each node in the syntax tree is transformed to obtain the corresponding intermediate representation.

[0083] For example, the global IR constructor maintains the insertion point, generates the IR according to the AST node order, and creates the main code block: the insertion point points to the basic block BB0; it processes the expression within parentheses: generating a temporary variable t1 = 2 - 1 / / the result is stored in the temporary variable t1; it processes multiplication: t2 = 5 * t1 / / using t1 from the previous step; it processes addition: t3 = 3 + t2 / / using t2 from the previous step; it processes assignment: a = t3 / / the final result is assigned to variable a. The generated IR code: BB0: t1 = SUB 2, 1 t2 = MUL 5, t1 t3 = ADD 3, t2 STORE a, t3 In this embodiment, an intermediate representation is generated by constructing an abstract syntax tree, which provides a data foundation for subsequent compilation.

[0084] In another embodiment of this application, the step of processing the intermediate representation through an assembler and a section allocator to obtain the binary file corresponding to the target platform includes: S51: The intermediate representation is read by the assembler to obtain the corresponding first machine instruction.

[0085] In this embodiment, the assembler generator is used to convert the intermediate representation into assembly language instructions. Assembly instructions are operators and mnemonics used in assembly language, used to correspond one-to-one with machine instructions. Each CPU has its own specific set of assembly instructions. Computers process data by executing instructions. An instruction typically contains two parts: an opcode (indicating the operation to be performed) and operands (indicating the source and destination of the data).

[0086] In this embodiment, the assembler reads the intermediate representation of the source code and converts each symbol in the intermediate representation into the first machine instruction of the assembly language for execution on the target platform.

[0087] S52: The section allocator generates the content of the corresponding section in the binary file according to the corresponding definition in the intermediate representation, thereby obtaining the binary file.

[0088] In this embodiment, a section allocator generates the corresponding content in the binary file based on the definitions in the intermediate representation, and then combines it with the generated first machine instructions to generate the binary file. This file can be run directly on the target platform.

[0089] For example, the target platform is a Linux system under x86_64, and the final output file of static compilation is an ELF64 format object file under Linux.

[0090] In this embodiment, the corresponding binary file is generated by the assembler and section allocator, ensuring that the source code runs on the target platform.

[0091] In another embodiment of this application, the step of compiling dynamic code using a real-time compilation engine to obtain the second machine instructions corresponding to the target platform includes: S61: When the dynamic code is a one-time executable code, the intermediate representation of the dynamic code is compiled through a preset interface to obtain the second machine instruction.

[0092] In this embodiment, one-time execution code is code that runs only once.

[0093] In this embodiment, when the real-time compilation engine compiles the code, a real-time compilation option needs to be added to the compilation command line. After adding the real-time compilation option, if the dynamic code is a one-time executable code, the string is obtained from the outside, and the preset interface is called to compile the string, so that the code can be run.

[0094] For example, the compiler includes a built-in runtime library for just-in-time (JIT) compilation. This library is based on LLVM and provides a JIT interface for developers. The runtime library supports two methods for using dynamic code: immediate execution and save-and-execute-on-demand. When building programs that require JIT compilation, the compiler needs to add the `jit` option to the compilation command line. After adding the `jit` option, the compiler will link the library when generating the binary executable file. For immediate execution scenarios, developers can obtain the code string from an external source and call the `jit` interface to run the code when dynamic compilation and execution are required.

[0095] S62: If the dynamic code is on-demand executable code, the intermediate representation of the dynamic code is compiled into a preset code module to obtain the second machine instruction.

[0096] In this embodiment, the code to be executed on demand is code that is compiled but not run directly; it is first stored in a dynamic link library and called when needed.

[0097] In this embodiment, for scenarios requiring on-demand execution after compilation, the code that needs to be compiled in real-time typically exposes a global function as the main entry point. Static-compiled code then calls this entry point function. Before the call, this function needs to be declared as an `extern JIT function`, meaning an externally compiled function in real-time. After this type of compilation, the source code is converted into a JIT code module and added to the runtime library's internal management data structure. During the call, it can be performed in the same way as a normal function. The compiler automatically adds implicit JIT calls and code to handle the return results, transforming these functions into calls to JIT functions.

[0098] In this embodiment, dynamically input code also needs to call external interfaces. Before calling an external interface, an external function needs to be declared in the dynamic code, and this external function needs to be implemented by the process running real-time compilation and exported to a dedicated symbol table. In practical application scenarios, most external calls are to other methods in the same code or static methods provided by the original process. Implementing a data engine that supports custom operators on data, the engine will provide some statically compiled application interfaces for dynamic code to call, while the operators come from external input. However, it is generally unlikely that one dynamically compiled operator will call another dynamically compiled operator code.

[0099] In this embodiment, the runtime library provides a mechanism for managing and retrieving entry points for static code. During compilation, the input parameters and return value types of the dynamic code need to be determined in advance. The static code then retrieves the required dynamic code based on the determined input parameters and return value types.

[0100] For example, the signature of a method is as follows (jitinf is a keyword used in the language): jitinf int64 cacl_method(int64[] array, uint64 n); The runtime library records a mapping in its memory structure: cacl_method corresponds to a set of methods that take an int64 array and uint64 as input and output an int64.

[0101] After that, it can be done in the code.

[0102] The code is registered using this method. This interface can compile the code and generate corresponding binary code. Finally, the code is executed as follows:

[0103] In this embodiment, dynamic code is compiled in two ways, enabling the compiler to modify the code during the execution of the process on the target platform without stopping the process, thus improving the system's operating efficiency.

[0104] In another embodiment of this application, the method further includes: S71: In the event of an error during compilation, record the location of the erroneous code, the cause of the error, and related variable information.

[0105] In this embodiment, if an error occurs during compilation, the compiler automatically records the location of the erroneous code, the cause of the error, and related variable information to remind the developer where the error occurred, making it easier for the developer to make adjustments and corrections.

[0106] In this embodiment, readable information is provided for parsing errors that occur during compilation, including the location of the erroneous code and the cause of the error. Because the causes of code errors are varied and may occur throughout the entire compilation process, error messages are made self-configurable for easy adjustment. A template file is provided, which includes source information for various compilation errors, such as error codes, line and column numbers, and variable name-related prompts. Users can add new source information to the template and rebuild the compiler.

[0107] In this embodiment, a custom template is used to set the original information of the mutation error, which ensures that errors that occur during compilation can be interpreted in a timely manner, thereby improving the efficiency and accuracy of compilation.

[0108] refer to Figure 2 , Figure 2 This is a schematic diagram of a compiler system proposed in an embodiment of this application, as shown below. Figure 2 As shown, in the static compilation process, an intermediate representation is obtained through a lexical analyzer and a semantic analyzer, which then generates static code and converts it into an executable file. In the dynamic compilation process, an intermediate representation is obtained through a lexical analyzer and a semantic analyzer, which then performs real-time compilation to obtain the corresponding execution result. Errors occurring during compilation are diagnosed using a diagnostic system.

[0109] refer to Figure 3 , Figure 3 This is a flowchart illustrating the lexical analysis state machine proposed in an embodiment of this application, as shown below. Figure 3As shown, the lexical analysis state machine reads each character in a loop, performs a judgment on each character, processes letters as identifiers, processes strings as strings, processes numbers as numbers, and processes comment symbols as comments, thereby converting the characters in the character stream into grammar symbols.

[0110] refer to Figure 4 , Figure 4 This is a code block jump relationship diagram after the if statement is transformed according to an embodiment of this application, such as... Figure 4 As shown, the current statement block in an if statement is called curBlock, thenBlock corresponds to the code executed when cond is true, elseBlock corresponds to the code executed when cond is false, and tailBlock corresponds to the code executed after the if statement ends.

[0111] In the embodiments described above, the programming language not only supports generating binary executable files but also possesses the ability to dynamically load and compile them. This broadens the application scope of the language and makes it more convenient for developers. When developers use the language to develop corresponding business logic, they can dynamically modify and run the code without downtime, and the efficiency reduction compared to code directly written in the source code is minimal. Because of the efficient execution of the code, request processing time and resource overhead are effectively reduced, thereby ensuring a normal user experience on the target platform.

[0112] It should be noted that the real-time compilation method provided in this application embodiment can be executed by a real-time compilation device, or a control module within that real-time compilation device for executing and loading the real-time compilation method. This application embodiment uses the execution of the real-time compilation method by a real-time compilation device as an example to illustrate the real-time compilation method provided in this application embodiment.

[0113] refer to Figure 5 , Figure 5 This is a schematic diagram of a real-time compilation apparatus 500 according to an embodiment of this application, as shown below. Figure 5 As shown, the device includes: The symbol set acquisition module 501 is used to convert the source code in the character stream into a symbol set through a lexical analysis state machine; The intermediate representation generation module 502 is used to process the grammar symbols in the symbol set through the analysis engine to generate the corresponding intermediate representation; The first machine instruction generation module 503 is used to process the intermediate representation through an assembler and a section allocator to obtain a binary file corresponding to the target platform. The binary file includes at least the first machine instruction corresponding to the target platform. The second machine instruction generation module 504 is used to compile dynamic code through a real-time compilation engine to obtain the second machine instruction corresponding to the target platform, and put the second machine instruction into a dynamic loading library to load and run the second machine instruction.

[0114] Optionally, the symbol set acquisition module includes: A character reading submodule is used to cyclically read characters from the character stream through the lexical analysis state machine; The character conversion submodule is used to perform state conversion based on the current state and the character, and to determine whether the sequence of characters conforms to preset syntax rules; The grammar symbol generation submodule is used to generate the grammar symbol corresponding to the character when the sequence of characters conforms to the preset grammar rules; The symbol set acquisition submodule is used to add the grammar symbol corresponding to each character to a set to obtain the symbol set.

[0115] Optionally, the device further includes: The verification module is used to identify the source code and determine whether the source code meets the preset identification conditions. The code saving module is used to save the source code to the character stream if the source code meets the preset recognition conditions.

[0116] Optionally, the intermediate representation generation module includes: The grammar symbol reading submodule is used to read the grammar symbols in the symbol set through the analysis engine; The abstract syntax tree construction submodule is used to construct the corresponding abstract syntax tree based on the grammar symbols using a recursive descent method. The abstract syntax tree includes multiple nodes and a corresponding list of basic code blocks. The intermediate representation generation submodule is used to transform each node in the abstract syntax tree to obtain the intermediate representation.

[0117] Optionally, the first machine instruction generation module includes: The first machine instruction generation submodule is used to read the intermediate representation through the assembler to obtain the corresponding first machine instruction; The binary file acquisition submodule is used to generate the content of the corresponding section in the binary file according to the corresponding definition in the intermediate representation through the section allocator, thereby obtaining the binary file.

[0118] Optionally, the second machine instruction generation module includes: A one-time code conversion submodule is used to compile an intermediate representation of the dynamic code through a preset interface to obtain the second machine instruction when the dynamic code is one-time executable code; The on-demand code conversion submodule is used to compile the intermediate representation of the dynamic code into a preset code module to obtain the second machine instruction when the dynamic code is on-demand code.

[0119] Optionally, the device further includes: The error logging submodule is used to record the location of the erroneous code, the cause of the error, and related variable information when an error occurs during compilation.

[0120] The real-time compilation device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0121] The real-time compilation device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0122] The real-time compilation device provided in this application embodiment can achieve... Figures 1 to 4 To avoid repetition, the various processes implemented by the real-time compilation device in any of the method embodiments will not be described again here.

[0123] Optionally, this application embodiment also provides an electronic device, including a processor 110, a memory 109, and a program or instructions stored in the memory 109 and executable on the processor 110. When the program or instructions are executed by the processor 110, they implement the various processes of the above-described real-time compilation method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0124] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0125] Figure 6 This is a schematic diagram of the hardware structure of an electronic device proposed in an embodiment of this application. The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0126] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here. This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described real-time compilation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0127] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0128] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described real-time compilation method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0129] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0130] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0132] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A real-time compilation method, characterized in that, The method includes: The source code in the character stream is converted into a symbol set by using a lexical analysis state machine; The analysis engine processes the grammar symbols in the symbol set to generate corresponding intermediate representations; The intermediate representation is processed by an assembler and a section allocator to obtain a binary file corresponding to the target platform. The binary file includes at least the first machine instructions corresponding to the target platform. The dynamic code is compiled by a real-time compilation engine to obtain the second machine instructions corresponding to the target platform, and the second machine instructions are put into a dynamic loading library to load and run the second machine instructions.

2. The real-time compilation method according to claim 1, characterized in that, The process of converting source code in a character stream into a symbol set using a lexical analysis state machine includes: The lexical analysis state machine reads characters from the character stream in a loop. The state transition is performed based on the current state and the characters, and it is determined whether the sequence of characters conforms to the preset syntax rules. If the sequence of characters conforms to a preset syntax rule, the corresponding grammar symbol for the character is generated; The grammar symbol corresponding to each character is added to the set to obtain the symbol set.

3. The real-time compilation method according to claim 1, characterized in that, Before converting the source code in the character stream into a symbol set through a lexical analysis state machine, the method further includes: The source code is identified to determine whether it meets preset identification conditions; If the source code meets the preset recognition conditions, the source code is saved to the character stream.

4. The real-time compilation method according to claim 1, characterized in that, The step of processing the symbols in the symbol set through the analysis engine to generate corresponding intermediate representations includes: The grammar symbols in the symbol set are read by the analysis engine; Based on the grammar symbols, a corresponding abstract syntax tree is constructed using a recursive descent method. The abstract syntax tree includes multiple nodes and a corresponding list of basic code blocks. Each node in the abstract syntax tree is transformed to obtain the intermediate representation.

5. The real-time compilation method according to claim 1, characterized in that, The intermediate representation is processed by an assembler and a section allocator to obtain the binary file corresponding to the target platform, including: The intermediate representation is read by the assembler to obtain the corresponding first machine instruction; The section allocator generates the content of the corresponding section in the binary file according to the corresponding definition in the intermediate representation, thereby obtaining the binary file.

6. The real-time compilation method according to claim 1, characterized in that, The process of compiling dynamic code using a real-time compilation engine to obtain the second machine instructions corresponding to the target platform includes: In the case that the dynamic code is one-time executable code, the intermediate representation of the dynamic code is compiled through a preset interface to obtain the second machine instruction; When the dynamic code is on-demand executable code, the intermediate representation of the dynamic code is compiled into a preset code module to obtain the second machine instruction.

7. The real-time compilation method according to claim 1, characterized in that, The method further includes: If an error occurs during compilation, record the location of the erroneous code, the cause of the error, and related variable information.

8. A real-time compilation apparatus, characterized in that, The device includes: The symbol set acquisition module is used to convert the source code in the character stream into a symbol set through a lexical analysis state machine; The intermediate representation generation module is used to process the grammar symbols in the symbol set through the analysis engine to generate corresponding intermediate representations; The first machine instruction generation module is used to process the intermediate representation through an assembler and a section allocator to obtain a binary file corresponding to the target platform. The binary file includes at least the first machine instruction corresponding to the target platform. The second machine instruction generation module is used to compile dynamic code through a real-time compilation engine to obtain the second machine instructions corresponding to the target platform, and put the second machine instructions into a dynamic loading library to load and run the second machine instructions.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of any of the methods described in claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of any of the methods described in claims 1-7.

Images