An antlr4-based code generator design method

Abstract

The application relates to an ANTLR4-based code generator design method and belongs to the field of code generation. The application generates a lexical analyzer and a syntax analyzer corresponding to an IDL syntax rule file; an IDL file is pretreated, the pretreated IDL file is generated into a syntax analysis tree through the lexical analyzer and the syntax analyzer, and the syntax analysis tree is further generated into structured data; a public interface file for generating C / C++, JAVA files is defined; the C / C++, JAVA public interface file reads a project template file, and structured data in the template file is replaced with structured data in step S2 to generate C / C++, JAVA project files. The application realizes the application of cross-platform and cross-language data distribution services.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of code generation, and particularly relates to a code generator design method based on antlr4. BACKGROUND

[0002] In order to improve the computing capacity of the domestic computing platform and adapt to the large amount of calculation in big data analysis, a distributed system has been widely applied due to its unique advantages. In the distributed system, the computing platform is responsible for the calculation of artificial intelligence deep learning algorithms, including image processing, voice analysis, pattern recognition, etc. The host realizes the scheduling of tasks, the reception and display of data. Data distribution service (DDS) is a kind of data transmission mode of publication and subscription, which not only defines a data-centered publication and subscription model, but also provides a data publication and subscription mechanism for the distributed system, and provides quality of service (QoS) services, meets the requirements of asynchronous and loose coupling in the distributed system, and guarantees the real-time performance of data distribution. The underlying code of DDS is realized through IDL (Interface Definition Language), and the user needs to define the interface of the data structure and attribute using IDL, and map the defined IDL interface to C / C++, Java and other commonly used programming languages through a code recognition tool, realize the isolation of the data interface and the programming language, and meet the requirements of cross-platform and cross-language of the distributed system.

[0003] Antlr4 is a powerful grammar analyzer tool, which can recognize, process and translate structured text, binary files and even XML files. He provides a framework that can contain Java, c++ action grammar description to construct language recognizer, compiler and parser. In antlr4, the user-defined context-free grammar file is parsed to automatically generate a lexical analyzer (Lexer) and a syntax analyzer (Parser), and the input text is automatically processed into a syntax analysis tree through a tree parser (Tree Parser). The automatic creation and traversal mechanism of the syntax analysis tree introduced by Antlr4 greatly improves the development efficiency of the language recognition program, and can free the developer from the complex compilation theory.

[0004] In order to improve the computing power of the domestic computing platform, to adapt to the demand of large amount of calculation in big data analysis, distributed system has been widely used with its unique advantages. In the distributed system, the computing platform is responsible for the calculation of artificial intelligence deep learning algorithm, including image processing, speech analysis, pattern recognition, etc. The host realizes the scheduling of tasks, the reception and display of data. There are many modes of data transmission in distributed system, such as point to point, message queue and publish subscribe data transmission mode. Among them, the time coupling degree of point to point data transmission mode is high, which makes the flexibility of data transmission poor and the real-time performance is not good; the data transmission mode of message queue solves the problem of time coupling through the way of message queue, but it also has the shortcomings of asynchronous communication, single point failure and poor reliability; in the data transmission mode of publish subscribe, the publisher and subscriber communicate through the topic, both sides only need to define the data structure and type of published data, without knowing the address of the other party and without being online at the same time, which has time and space decoupling.

[0005] Data distribution service (DDS) is a kind of publish subscribe data transmission mode, which not only defines the data centered publish subscribe model and provides the data publish subscribe mechanism for distributed system, but also provides the quality of service (QoS) service, which meets the requirements of asynchronous and loose coupling in distributed system and guarantees the real-time performance of data distribution. The underlying code of DDS is realized through IDL (Interface Definition Language), users need to define the interface of data structure and attribute using IDL, and map the defined IDL interface to C / C++, Java and other commonly used programming languages through code recognition tool, which realizes the isolation of data interface and programming language and meets the requirements of cross platform and cross language in distributed system. By using the publish / subscribe mode, DDS establishes an abstract communication between the data sender and receiver, the data publisher does not need to know the address of each receiver, it only needs to publish the data to be published according to a certain data type. Similarly, after the subscriber subscribes to the interested data, it does not need to know where the published data comes from, it only needs to know the data type of the data it wants to receive, and it can get the data it wants.

[0006] The existing code generator has complex functions and large compiled project files, which can realize the mapping of IDL to C / C++, Java and other programming languages, but the generated project files can only be compiled and run in a specific compilation environment. The code generator designed based on antl4 can convert the data interface type defined by IDL into CMake file, the generated file is small, and it has good adaptability to the compilation environment, which can realize the compilation and running in different platforms such as Windows / Linux, and solves the problem of cross platform and cross language data transmission in distributed system. SUMMARY

[0007] (One) technical problems to be solved

[0008] The technical problem to be solved by the present application is how to provide an antlr4-based code generator design method to solve the problem of cross-platform and cross-language data transmission in distributed systems.

[0009] (Two) technical solutions

[0010] In order to solve the above technical problems, the present application provides an antlr4-based code generator design method, which comprises the following steps:

[0011] S1: generating a lexical analyzer and a syntax analyzer corresponding to an IDL syntax rule file;

[0012] S2: preprocessing the IDL file, generating a syntax analysis tree from the preprocessed IDL file through the lexical analyzer and the syntax analyzer, and further generating structured data from the syntax analysis tree;

[0013] S3: defining a common interface file for generating C / C++ and JAVA files;

[0014] S4: the C / C++ and JAVA common interface file reads a project template file, and replaces the structured data in the template file with the structured data in step S2 to generate C / C++ and JAVA project files.

[0015] Further, the step S1 specifically comprises:

[0016] S11, defining an IDL syntax rule file;

[0017] S12, generating a lexical analyzer and a code syntax analyzer for analyzing the IDL file through the antlr4 tool;

[0018] S13, obtaining the ALTAR4-RUNTIME library called by the runtime of the lexical analyzer and the syntax analyzer.

[0019] Further, the IDL syntax rule file is a.g4 file.

[0020] Further, the ANTL-RUNTIME is provided by antlr4 middleware and stored in the antlr4 directory, which contains antlr4-runtime source code.

[0021] Further, the step S2 specifically comprises the following steps:

[0022] S21, inputting a user-defined IDL file;

[0023] S22, the code preprocessing module processes the preprocessing instruction in the IDL file; the include file processing module is responsible for loading and processing the including code in the IDL file;

[0024] S23, the processed IDL file is generated into a corresponding syntax analysis tree through a lexical analyzer and a syntax analyzer;

[0025] S24, the generated syntax analysis tree is further processed by a structured data generation module to form a friendly structured data.

[0026] Further, the preprocessing instruction includes #ifdef, #define, #undef, #else and #endif; the including code includes #include "xxxx.IDL".

[0027] Further, the step S3 specifically includes:

[0028] S31, the source file public interface generation module is provided by antlr4, which is a base class of C / C++ and JAVA file public interface, and defines the public interface and attributes required for code file generation;

[0029] S32, based on the source file public interface generation module, a public interface file for generating C / C++ and JAVA files is defined, and the C / C++ and JAVA file public interface generation module abstracts and defines the public interface file for generating C / C++ and JAVA files according to the characteristics of C / C++ and JAVA languages.

[0030] Further, the C / C++ and JAVA file public interface file includes Type, Type_Plugin, Type_Support, Type_Publisher, Type_Subscriber and CMakeList.

[0031] Further, the step S4 specifically includes:

[0032] S41, based on the interface rule of DDS, an engineering template file for generating C / C++ and JAVA is defined;

[0033] S42, the C / C++ and JAVA public interface file reads the engineering template file, and replaces the structured data in the template file with the structured data in step S2 to generate a corresponding C / C++ and JAVA engineering file, which includes a Publisher source file, a Subscriber source file and a CMakeList file.

[0034] Further, the step S4 further comprises: copying the C / C++, JAVA project file to the Windows / Linux environment, directly using the CMake command to build the project, and compiling to generate an executable file.

[0035] (III) Beneficial Effects

[0036] The application proposes an antlr4-based code generator design method, which stores a user-defined data structure type of a subscription / publishing data distribution service (DDS) as an IDL file as an input of the code generator, obtains a CMake file through the code generator, generates a project file through a CMake command in a Windows / Linux environment, and generates an executable file through a compiler in the Windows / Linux environment, so as to realize application of the cross-platform and cross-language data distribution service. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The flow process of antlr4 code parsing of the application;

[0038] Figure 2 The software structure block diagram of the code generator of the application;

[0039] Figure 3 The workflow diagram of the code generator of the application;

[0040] Figure 4 The GUI interface of the code generator of the application. DETAILED DESCRIPTION

[0041] In order to make the purpose, content and advantages of the application more clear, the specific implementation of the application is described in further detail below in combination with the drawings and examples.

[0042] The application proposes an antlr4-based code generator, which can generate a CMake script file from a data interface type defined by an IDL, so as to realize compilation and running in different platforms such as Windows / Linux, and solve the problem of cross-platform and cross-language data transmission of a distributed system.

[0043] The application proposes an antlr4-based code generator, which stores a user-defined data structure type of a subscription / publishing data distribution service (DDS) as an IDL file as an input of the code generator, obtains a CMake file through the code generator, generates a project file through a CMake command in a Windows / Linux environment, and generates an executable file through a compiler in the Windows / Linux environment, so as to realize application of the cross-platform and cross-language data distribution service.

[0044] As Figure 1 The application provides an antlr4-based code generator, which provides a framework to automatically generate a lexical analyzer (Lexer), a syntax analyzer (Parser) and a syntax analysis tree (TreeParser) according to a user-defined grammar file, wherein the lexical analyzer generates lexical symbol tokens from input text and the lexical symbol tokens are input to the syntax analyzer, and the syntax analyzer outputs a syntax analysis tree.

[0045] The antlr4-based code generator of the application saves a user-defined data structure type of a subscription / publishing data distribution service (DDS) as an IDL file as input of the code generator, obtains an engineering file through the code generator, builds the engineering file using a CMake command in a Windows / Linux environment, and compiles to generate an executable file, so as to realize a cross-platform and cross-language data distribution service.

[0046] The code generator is divided into four steps in terms of design structure: generating a lexical and syntax analyzer, generating a syntax analysis tree and converting the syntax analysis tree into structured data, defining a public interface file for generating C / C++ and JAVA files, and generating C / C++ and JAVA engineering files, and the specific steps are as follows:

[0047] S1: generating a lexical analyzer and a syntax analyzer corresponding to an IDL syntax rule file

[0048] S11, defining an IDL syntax rule file (.g4 file);

[0049] S12, generating a lexical analyzer and a code syntax analyzer for analyzing an IDL file through an antlr4 tool;

[0050] S13, the lexical analyzer and the syntax analyzer call an ALTAR4-RUNTIME library during runtime, the ANTLR-RUNTIME is provided by antlr4 middleware and stored in an antlr4 directory, and the ANTLR-RUNTIME contains antlr4-runtime source code.

[0051] S2: preprocessing an IDL file, generating a syntax analysis tree from the preprocessed IDL file through a lexical analyzer and a syntax analyzer, and further generating structured data from the syntax analysis tree

[0052] S21, inputting a user-defined IDL file;

[0053] S22, the code preprocessing module processes the preprocessing instructions in the IDL file, including #ifdef, #define, #undef, #else, #endif and other instructions; the include file processing module is responsible for loading and processing the included code in the IDL file, such as #include "xxxx.IDL";

[0054] S23, the processed IDL file is generated into a corresponding syntax analysis tree through a lexical analyzer and a syntax analyzer;

[0055] S24, the structured data generation module further processes the generated syntax analysis tree to form friendly structured data.

[0056] S3: define the public interface file for generating C / C++, JAVA files

[0057] S31, the source file public interface generation module is provided by antlr4, which is the base class of the C / C++, JAVA file public interface, and defines the public interface and attributes required for code file generation;

[0058] S32, based on the source file public interface generation module, define the public interface file for generating C / C++, JAVA files, the C / C++, JAVA file public interface generation module abstracts and defines the public interface file for generating C / C++, JAVA files according to the characteristics of C / C++, JAVA language, mainly including Type, Type_Plugin, Type_Support, Type_Publisher, Type_Subscriber, CMakeList and the like.

[0059] S4: the C / C++, JAVA public interface file reads the project template file, and replaces the structured data in the template file with the structured data in step S2 to generate C / C++, JAVA project files

[0060] S41, based on the interface rules of DDS, define the project template file for generating C / C++, JAVA;

[0061] S42, the C / C++, JAVA public interface file reads the project template file, and replaces the structured data in the template file with the structured data in step S2 to generate corresponding C / C++, JAVA project files; which includes Publisher source file, Subscriber source file, CMakeList file and other files.

[0062] Copy C / C++, JAVA project files to different environments such as Windows / Linux, etc., can directly use CMake command to build the project, and compile to generate executable files, so as to realize the application of cross-platform and cross-language data distribution service.

[0063] Wherein, the GUI (user interface) provides a friendly, easy-to-use, cross-platform interactive interface for users through Qt5 as an interface development tool; the auxiliary module provides function interfaces for string operation and program command line parsing functions.

[0064] The error information existing in the IDL code parsing process is output to the console by the exception processing module.

[0065] The above only describes the preferred embodiments of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should be considered as the protection scope of the present application.

Claims

1. An antlr4-based code generator design method, characterized in that, The method comprises the following steps: S1: generating a lexical analyzer and a syntax analyzer corresponding to an IDL syntax rule file; S2: preprocessing an IDL file, generating a syntax analysis tree from the preprocessed IDL file through the lexical analyzer and the syntax analyzer, and further generating structured data from the syntax analysis tree; S3: defining a common interface file for generating C / C++, JAVA files; S4: reading an engineering template file by the C / C++, JAVA common interface file, and replacing structured data in the template file with the structured data in step S2 to generate a C / C++, JAVA engineering file; Wherein, The step S4 specifically comprises: S41, based on the interface rule of DDS, defining an engineering template file for generating C / C++, JAVA; S42, the C / C++, JAVA common interface file reads the engineering template file, and replaces the structured data in the template file with the structured data in step S2 to generate a corresponding C / C++, JAVA engineering file, which includes a Publisher source file, a Subscriber source file and a CMakeList file.

2. The antlr4-based code generator design method of claim 1, wherein, The step S1 specifically comprises: S11, defining an IDL syntax rule file; S12, generating a lexical analyzer and a code syntax analyzer for analyzing an IDL file through an antlr4 tool; S13, acquiring an ALTAR4-RUNTIME library called by a runtime of the lexical analyzer and the syntax analyzer.

3. The antlr4-based code generator design method of claim 2, wherein, The IDL syntax rule file is a.g4 file.

4. The antlr4-based code generator design method of claim 2, wherein, The ANTLR-RUNTIME is provided by antlr4 middleware and stored in an antlr4 directory, which contains antlr4-runtime source code.

5. The antlr4-based code generator design method of any one of claims 2-4, wherein, The step S2 specifically comprises the following steps: S21, inputting a user-defined IDL file; S22, a code preprocessing module processes preprocessing instructions in the IDL file; a file inclusion processing module is responsible for loading and processing inclusion codes in the IDL file; S23, generating a corresponding syntax analysis tree from the processed IDL file through the lexical analyzer and the syntax analyzer; S24, a structured data generation module further processes the generated syntax analysis tree to form friendly structured data.

6. The antlr4-based code generator design method of claim 5, wherein, The preprocessing instructions include#ifdef, #define, #undef, #else and #endif; and the inclusion codes include#include"xxxx.IDL”.

7. The antlr4-based code generator design method of claim 5, wherein, The step S3 specifically comprises: S31, a source file common interface generation module is provided by antlr4, which is a base class of C / C++, JAVA file common interfaces, and defines common interfaces and attributes required for code file generation; S32, based on the source file common interface generation module, a common interface file for generating C / C++, JAVA files is defined, and the C / C++, JAVA file common interface generation module abstracts and defines the common interface file for generating C / C++, JAVA files according to the characteristics of C / C++, JAVA languages.

8. The antlr4-based code generator design method of claim 7, wherein, The public interface file of the C / C++, JAVA file includes Type, Type_Plugin, Type_Support, Type_Publisher, Type_Subscriber and CMakeList.

9. The antlr4-based code generator design method of claim 8, wherein, The step S4 further includes copying the C / C++, JAVA project file to the Windows / Linux environment, directly using the CMake command to build the project, and compiling to generate an executable file.

Images