DSL-based common dimension data conversion method and system

Abstract

The invention provides a DSL (Digital Subscriber Language)-based public dimension data conversion method and system, and the method comprises the following steps: reading an input script as a file stream by using a business language script reading module; segmenting characters in the file stream into tokens by utilizing a lexical analysis module; using a grammar parser module to construct the token into a grammar tree; traversing all nodes of the syntax tree by using a syntax traversal module, and triggering a conversion rule monitor module at each node; and extracting information by using the conversion rule monitor module, and assembling the information into a computer language program. According to the technical scheme, modeling is carried out on the service function through the configuration data conversion assembly of the DSL, the model is formed, an existing model or a newly-built model is introduced during modeling, and model verification is carried out on a system, so that the condition of repeated modeling is avoided, and a bloated model can be simplified.

Description

Background Art

[0002] The current manual development of data scripts will lead to data redundancy and stovepipe construction (i.e. each system or department independently builds its own data processing flow, resulting in data silos and duplication of work) during the data warehouse construction process. These problems not only increase maintenance costs, but also reduce data consistency and availability.

[0003] There are mainly the following pain points:

[0004] 1) Low efficiency

[0005] Developers need to rewrite scripts every time they handle the same data change scenario. At the same time, after too much script code accumulates, there is little reuse of existing code, and it is impossible to respond quickly to business demands.

[0006] 2) High risk

[0007] Since the script code is completely written manually, once the responsible personnel are changed, the original logic cannot be traced when the modification is made, which can easily lead to data anomalies or even production accidents, resulting in increased data security risks.

[0008] 3) Non-standard operations

[0009] Since manual script writing lacks the constraints of standardized tools, the scripts written by each person are not uniform in style, difficult to standardize, and have varying quality. The quality of the code is determined by the quality of the developer, which brings high management costs and is prone to production problems.

[0010] 4) High communication costs

[0011] During the process of changing requirements, communication between IT and business personnel is time-consuming. There is a situation where IT personnel use technical terms while business personnel use business terms, which results in high communication costs. Summary of the invention

[0012] The present application provides a DSL-based public dimension data conversion method and system to reduce the maintenance cost of data-based script programs and improve data consistency and availability.

[0013] In a first aspect, a DSL-based public dimension data conversion method is provided, comprising the following steps:

[0014] Using the business language script reading module, the input script is read as a file stream, and the obtained file stream is input into the lexical analysis module;

[0015] Using a lexical analysis module to segment characters in the file stream into tokens, and inputting the obtained tokens into a syntax analyzer module;

[0016] Using a syntax parser module to construct the token into a syntax tree;

[0017] Using a grammar traversal module, traverse all nodes of the grammar tree, and trigger a conversion rule listener module at each node;

[0018] The information is extracted using the transformation rule listener module and assembled into a computer language program.

[0019] In the above technical solution, the business language script reading module is used to read the input script as a file stream, and the obtained file stream is input into the lexical parsing module; the lexical parsing module is used to segment the characters in the file stream into tokens, and the obtained tokens are input into the syntax parser module; the syntax parser module is used to construct the tokens into a syntax tree; the syntax traversal module is used to traverse all nodes of the syntax tree, and the conversion rule listener module is triggered at each node; the conversion rule listener module is used to extract information and assemble it into a computer language program; through the componentized design principle, the code template is abstracted to form a common macro, each macro module is relatively independent, and the macro can be referenced in the code to reduce the amount of code; the business function is modeled by the configurable data conversion component of the DSL to form a model, and an existing model is introduced or a new model is created during modeling. The system performs model verification to avoid repeated modeling, which can simplify the bloated model; the parsed DSL is formed into metadata, stored in the database, and the metadata is used to build functions such as asset management, lineage display, indicator tracking, and attribution analysis.

[0020] In a specific embodiment, it also includes:

[0021] The computer language program is executed using the conversion encapsulation module, and finally the information is saved and the associated function is triggered.

[0022] In a specific implementation scheme, the steps of using a lexical parsing module to segment the characters in the file stream into tokens and inputting the obtained tokens into a syntax parser module are specifically as follows:

[0023] The lexical analysis module is used to segment the characters in the file stream into tokens according to predefined lexical rules, and the obtained tokens are input into the syntax analyzer module.

[0024] In a specific implementation scheme, the predefined lexical rules include keywords and reserved words, identifiers, operators and separators, constants, comments, blank characters, string literals, character literals, and regular expressions.

[0025] In a specific implementation scheme, the step of constructing the token into a syntax tree using a syntax parser module is specifically as follows:

[0026] The token is constructed into a syntax tree according to predefined syntax rules using a syntax parser module.

[0027] In a specific implementation scheme, the predefined grammar rules include context-free grammar, rule forms, recursive rules, precedence and associativity, error handling, language-specific features, and example rules.

[0028] In a specific implementation scheme, the steps of extracting information using the conversion rule listener module and assembling it into a computer language program are as follows:

[0029] Using the transformation rule listener module, information is extracted according to the defined transformation rules and assembled into a computer language program.

[0030] In a specific implementation scheme, the conversion rule listener module is used to extract information according to the defined conversion rules and assemble the information into a computer language program, which specifically includes:

[0031] Define transformation rules;

[0032] Design the transformation rule listener module;

[0033] Build conversion logic;

[0034] Configure the conversion rules and deploy the listener.

[0035] In a specific implementation scheme, the defined conversion rules include the structure of the source data, the format of the target data, and specific steps on how to extract information from the source data and convert it into the target data.

[0036] In a second aspect, a DSL-based public dimension data conversion system is provided, including:

[0037] A business language script reading module, used for reading the input script as a file stream, and inputting the obtained file stream into the lexical parsing module;

[0038] A lexical analysis module, used for dividing the characters in the file stream into tokens, and inputting the obtained tokens into a syntax analyzer module;

[0039] A syntax parser module, used to construct the token into a syntax tree;

[0040] A grammar traversal module, used for traversing all nodes of the grammar tree and triggering a conversion rule listener module at each node;

[0041] A conversion rule listener module is used to extract information and assemble it into a computer language program;

[0042] The conversion and encapsulation module is used to execute the computer language program and finally complete the storage of information and the triggering of associated functions.

[0043] In the above technical solution, the business language script reading module is used to read the input script as a file stream, and the obtained file stream is input into the lexical parsing module; the lexical parsing module is used to segment the characters in the file stream into tokens, and the obtained tokens are input into the syntax parser module; the syntax parser module is used to construct the tokens into a syntax tree; the syntax traversal module is used to traverse all nodes of the syntax tree, and the conversion rule listener module is triggered at each node; the conversion rule listener module is used to extract information and assemble it into a computer language program; through the componentized design principle, the code template is abstracted to form a common macro, each macro module is relatively independent, and the macro can be referenced in the code to reduce the amount of code; the business function is modeled by the configurable data conversion component of the DSL to form a model, and an existing model is introduced or a new model is created during modeling. The system performs model verification to avoid repeated modeling, which can simplify the bloated model; the parsed DSL is formed into metadata, stored in the database, and the metadata is used to build functions such as asset management, lineage display, indicator tracking, and attribution analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a DSL-based public dimension data conversion method provided in an embodiment of the present application;

[0045] Figure 2 A structural block diagram of a DSL-based public dimension data conversion system provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a DSL-based public dimension data conversion system provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of metadata provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.

[0049] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0050] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0051] To facilitate understanding of the DSL-based public dimension data conversion method and system provided in the embodiment of the present application, its application scenario is first explained. The DSL-based public dimension data conversion method and system provided in the embodiment of the present application are used to reduce the maintenance cost of data-based script programs and improve the consistency and availability of data. In the current manual development of data-based script programs, during the construction of the data warehouse, as the system construction time goes by, problems such as data redundancy and chimney construction (i.e., each system or department independently builds its own data processing process, resulting in data islands and duplication of work) will be encountered. These problems not only increase the maintenance cost, but also reduce the consistency and availability of data. There are mainly the following pain points: 1) Low efficiency, developers need to rewrite the script every time they deal with the same number change scenario, and after the script code accumulates too much, there are fewer cases of reusing existing codes, and it is impossible to respond quickly to business demands. 2) High risk, because the script code is completely written by hand, after the responsible personnel are changed, the original logic cannot be traced when the modification is made, which is easy to cause data anomalies and even production accidents, resulting in increased data security risks. 3) Non-standardized operations. Since the manual script writing method lacks the constraints of standardized tools, the scripts written by each person are not uniform in style, difficult to standardize, and have uneven quality. The quality of the code is determined by the developer's literacy, which brings high management costs and is prone to production problems. 4) High communication costs. During the process of changing requirements, communication between IT and business personnel is time-consuming. There are situations where IT uses technical terms and business uses business terms, and the communication cost is high. For this reason, the embodiment of the present application provides a public dimension data conversion method and system based on DSL to reduce the maintenance cost of data-based script programs and improve data consistency and availability. The following is a detailed description of the embodiments in conjunction with specific drawings.

[0052] refer to Figures 1 to 4 , Figure 1 A flowchart of a DSL-based public dimension data conversion method provided in an embodiment of the present application; Figure 2 A structural block diagram of a DSL-based public dimension data conversion system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a DSL-based public dimension data conversion system provided in an embodiment of the present application; Figure 4A schematic diagram of the structure of metadata provided in an embodiment of the present application.

[0053] exist Figure 1 In the embodiment of the present application, a method for converting public dimension data based on DSL is provided, comprising the following steps:

[0054] Using the business language script reading module, the input script is read as a file stream, and the obtained file stream is input into the lexical analysis module;

[0055] Using a lexical analysis module to segment characters in the file stream into tokens, and inputting the obtained tokens into a syntax analyzer module;

[0056] Using a syntax parser module to construct the token into a syntax tree;

[0057] Using a grammar traversal module, traverse all nodes of the grammar tree, and trigger a conversion rule listener module at each node;

[0058] The information is extracted using the transformation rule listener module and assembled into a computer language program.

[0059] In the above technical solution, the business language script reading module is used to read the input script as a file stream, and the obtained file stream is input into the lexical parsing module; the lexical parsing module is used to segment the characters in the file stream into tokens, and the obtained tokens are input into the syntax parser module; the syntax parser module is used to construct the tokens into a syntax tree; the syntax traversal module is used to traverse all nodes of the syntax tree, and the conversion rule listener module is triggered at each node; the conversion rule listener module is used to extract information and assemble it into a computer language program; through the componentized design principle, the code template is abstracted to form a common macro, each macro module is relatively independent, and the macro can be referenced in the code to reduce the amount of code; the business function is modeled by the configurable data conversion component of the DSL to form a model, and an existing model is introduced or a new model is created during modeling. The system performs model verification to avoid repeated modeling, which can simplify the bloated model; the parsed DSL is formed into metadata, stored in the database, and the metadata is used to build functions such as asset management, lineage display, indicator tracking, and attribution analysis.

[0060] In a specific embodiment, it also includes:

[0061] The computer language program is executed using the conversion encapsulation module, and finally the information is saved and the associated function is triggered.

[0062] In a specific implementation scheme, the steps of using a lexical parsing module to segment the characters in the file stream into tokens and inputting the obtained tokens into a syntax parser module are specifically as follows:

[0063] The lexical analysis module is used to segment the characters in the file stream into tokens according to predefined lexical rules, and the obtained tokens are input into the syntax analyzer module.

[0064] It should be noted that in computer science and natural language processing (NLP), "token" is a basic unit, which usually refers to the element obtained after segmentation in text or data stream. In the process of segmenting characters into tokens, tokens can be words, punctuation marks, numbers, special symbols, or any text fragments defined according to specific needs.

[0065] Tokenization is a basic step in natural language processing, which involves dividing the input text (such as sentences or paragraphs) into smaller, meaningful units (i.e. tokens). These tokens are usually used for subsequent text analysis tasks such as part-of-speech tagging (POS Tagging), syntactic analysis (Parsing), named entity recognition (Named Entity Recognition, NER), etc.

[0066] In different contexts and applications, the specific definition of token may vary. For example:

[0067] In English text processing, a token may be a word, punctuation mark (such as comma, period, question mark, etc.) or number.

[0068] In some specific NLP tasks, a token may be a subword unit, which is achieved by further splitting the word into smaller, meaningful fragments, especially useful when dealing with Out-of-Vocabulary (OOV).

[0069] In programming languages ​​and source code analysis, a token may be a keyword, identifier, operator, or punctuation mark, etc.

[0070] Tokenization is a core step in many fields such as natural language processing, text mining, information retrieval, compiler design, etc. It is essential for understanding and analyzing text data. The accuracy and efficiency of subsequent processing steps can be significantly improved through the appropriate tokenization strategy.

[0071] In a specific implementation scheme, the predefined lexical rules include keywords and reserved words, identifiers, operators and separators, constants, comments, blank characters, string literals, character literals, and regular expressions.

[0072] Specifically, in the step of segmenting characters in a file stream into tokens, the predefined lexical rules usually include the following aspects:

[0073] 1. Keywords and reserved words

[0074] Definition: Keywords are predefined words with special meanings in programming languages, such as if, else, for, etc. Reserved words are those that may not be used directly as keywords but may be used as keywords in future versions.

[0075] Rules: The lexical analyzer needs to recognize these keywords and reserved words and output them as separate tokens.

[0076] 2. Identifier

[0077] Definition: An identifier is a user-defined name used to name elements such as variables, functions, and arrays.

[0078] Rules: Generally, identifiers are composed of letters, numbers, and underscores (_), and cannot start with a number. The lexical analyzer needs to recognize identifiers according to these rules.

[0079] 3. Operators and delimiters

[0080] Definition: Operators are used to perform arithmetic operations (such as +, -, *, / ), logical operations (such as &&, ||), assignments (such as =), etc. Delimiters are used to separate different parts of the code, such as commas (,), semicolons (;), parentheses (()), etc.

[0081] Rules: The lexical analyzer needs to recognize these single-character or multi-character operators and delimiters and output them as separate tokens.

[0082] 4. Constants

[0083] Definition: A constant is a quantity whose value does not change during program execution, such as numbers (integers, floating-point numbers), string literals, etc.

[0084] Rules: The lexical analyzer needs to recognize these constants and output them as separate tokens according to their type (such as integer, floating point number, string).

[0085] 5. Annotations

[0086] Definition: Comments are explanatory text that programmers add to code to explain the function or purpose of the code.

[0087] Rule: The lexical analyzer needs to recognize comments and ignore them during processing. The recognition of comments usually relies on specific grammatical markers, such as / / (single-line comment) and / *...* / (multi-line comment) in C language.

[0088] 6. Whitespace

[0089] Definition: Whitespace characters include spaces, tabs, and newline characters. They are used to separate different elements in the code, but they do not carry any semantic information themselves.

[0090] Rule: The lexical analyzer needs to ignore these whitespace characters as they have no direct effect on the execution of the program.

[0091] 7. String literals

[0092] Definition: A string literal is a sequence of characters enclosed in quotes (single or double, depending on the programming language).

[0093] Rules: The lexical analyzer needs to recognize string literals and output them as separate tokens. At the same time, it also needs to handle escape characters in the string (such as \n for line breaks).

[0094] 8. Character literals

[0095] Definition: A character literal is a single character enclosed in single quotes.

[0096] Rules: The lexical analyzer needs to recognize character literals and output them as separate tokens. At the same time, it also needs to handle escape characters in character literals (such as \' represents the single quote character itself).

[0097] 9. Regular Expressions

[0098] Definition: Regular expressions are a powerful text processing tool for matching specific patterns in strings.

[0099] Rules: In the lexical analyzer, you can use regular expressions to describe complex lexical rules. For example, you can use regular expressions to match identifiers, numbers, string literals, etc.

[0100] In summary, the predefined lexical rules are the basis for the lexical analyzer to split characters in the file stream into tokens. These rules cover keywords and reserved words, identifiers, operators and separators, constants, comments, whitespace characters, string literals, character literals, and the use of regular expressions.

[0101] In a specific implementation scheme, the step of constructing the token into a syntax tree using a syntax parser module is specifically as follows:

[0102] The token is constructed into a syntax tree according to predefined syntax rules using a syntax parser module.

[0103] In a specific implementation scheme, the predefined grammar rules include context-free grammar, rule forms, recursive rules, precedence and associativity, error handling, language-specific features, and example rules.

[0104] Specifically, in the step of constructing a token into a syntax tree according to predefined syntax rules, the predefined syntax rules generally include the following aspects:

[0105] 1. Context-Free Grammar (CFG)

[0106] Definition: A context-free grammar is a method for formally describing the grammatical structure of a language. It consists of a set of production rules that describe how to generate strings from non-terminal symbols (representing grammatical structures) and terminal symbols (representing tokens).

[0107] Application: In compiler design, CFG is widely used to define the grammatical rules of programming languages. Through these rules, the parser can recognize grammatical structures and construct the corresponding syntax tree.

[0108] 2. Form of rules

[0109] Rules usually come in the form of "productions", where a non-terminal symbol can be replaced by a sequence of one or more terminals and non-terminals.

[0110] For example, in C language, the non-terminal symbol "expression" can be composed of structures such as "terms" and "operator terms", where "terms" can be composed of "factors" in a specific way.

[0111] 3. Recursive rules

[0112] Context-free grammars allow recursion in rules, that is, a non-terminal symbol can appear again in its production rule.

[0113] Recursive rules are the basis for building complex grammatical structures (such as nested function calls, loop statements, etc.).

[0114] 4. Priority and Associativity

[0115] In some cases, the predefined grammar rules also need to take into account operator precedence and associativity.

[0116] Precedence determines which operator should be applied first when multiple operators appear in the same expression.

[0117] Associativity determines how operators with the same precedence should group their operands.

[0118] 5. Error handling

[0119] Predefined grammar rules usually also include error handling mechanisms to detect and report syntax errors during the parsing process.

[0120] These mechanisms may include error recovery strategies (such as skipping certain tokens to continue parsing), error reporting (providing detailed information about the location and nature of the error), etc.

[0121] 6. Language-specific features

[0122] Different programming languages ​​have different grammatical features, so the predefined grammar rules also vary from language to language.

[0123] For example, some languages ​​may support advanced features such as anonymous functions, generics, pattern matching, etc., which need to be defined accordingly in the grammar rules.

[0124] 7. Example Rules

[0125] The following is a simplified example rule to illustrate how to build a syntax tree based on predefined grammar rules:

[0126] Suppose there is a simple expression language that supports integers, the addition operator (+), and parentheses to group expressions.

[0127] Define the following production rules:

[0128] <expression>::=<term>|<expression>+<term>

[0129] <item>::=integer|(<expression>)

[0130] According to these rules, the parser can construct the input token sequence (such as "3+4*(5-2)") into the corresponding syntax tree.

[0131] In short, predefined grammar rules are the basis for building a syntax tree. They define how to combine tokens into grammatical structures and take into account factors such as operator precedence, associativity, recursive rules, and specific language characteristics. By following these rules, the parser can accurately build a syntax tree that reflects the structure of the source code.

[0132] In a specific implementation scheme, the steps of extracting information using the conversion rule listener module and assembling it into a computer language program are as follows:

[0133] Using the transformation rule listener module, information is extracted according to the defined transformation rules and assembled into a computer language program.

[0134] In a specific implementation scheme, the conversion rule listener module is used to extract information according to the defined conversion rules and assemble the information into a computer language program, which specifically includes:

[0135] Define transformation rules;

[0136] Design the transformation rule listener module;

[0137] Build conversion logic;

[0138] Configure the conversion rules and deploy the listener.

[0139] Specifically, the specific steps of using a transformation rule listener module (usually a system component designed to monitor and trigger transformation operations) to extract information according to defined transformation rules and assemble it into a computer language program include:

[0140] 1. Define conversion rules

[0141] First, all transformation rules need to be clearly defined. These rules should clearly describe the structure of the source data, the format of the target data, and the specific steps of how to extract information from the source data and transform it into the target data.

[0142] 2. Design the conversion rule listener module

[0143] Interface definition: Define the interface for the conversion rule listener module, including registering conversion rules, triggering conversion operations, processing conversion results, etc.

[0144] Rule registration: Implement a mechanism to register transformation rules. This usually involves storing the rules in some form of data structure, such as a list, dictionary, or database, so that they can be easily retrieved when needed.

[0145] Event Listening: Design listeners to listen for specific events or data changes that will trigger the execution of transformation operations.

[0146] 3. Implement conversion logic

[0147] Transformation Functions: Write or implement one or more transformation functions for each transformation rule. These functions are responsible for extracting information from the source data, applying the transformation rules, and assembling the results into the target format.

[0148] Integration Listener: Integrate the conversion function with the conversion rule listener module so that the corresponding conversion function can be called when the event is triggered.

[0149] 4. Configuration and deployment

[0150] Configure conversion rules: Configure conversion rules in the system, including specifying the path and format of source data and target data, and registering corresponding conversion functions.

[0151] Deploy the listener: Deploy the transformation rule listener module to the appropriate environment and ensure that it can listen to the events or data changes that need to trigger the transformation operation.

[0152] 5. Trigger and conversion

[0153] Event triggering: When predefined events or data changes occur, the transformation rule listener module will capture these events and trigger corresponding transformation operations according to the configured transformation rules.

[0154] Execute transformations: The transformation rule listener module calls the registered transformation functions, extracts information from the source data, applies the transformation rules, and assembles the results into the target format.

[0155] 6. Process the conversion results

[0156] Result Validation: After the conversion is complete, the results are validated to ensure that they meet the expected format and quality standards.

[0157] Result storage: Store the converted data to a target location, such as a file, database, or other storage system.

[0158] 7. Monitoring and logging

[0159] Monitoring: Set up a monitoring mechanism to track the performance and status of the conversion process so that potential problems can be discovered and resolved in a timely manner.

[0160] Logging: Record key information during the conversion process, such as conversion time, successful or failed conversion operations, exception information, etc., to facilitate problem tracking and performance optimization.

[0161] 8. Maintenance and Updates

[0162] Regular maintenance: Regularly check the performance and stability of the conversion rule listener module, and perform necessary maintenance and optimization.

[0163] Update rules: Update transformation rules based on changes in business requirements or data sources, and ensure that the transformation rule listener module can handle these updates correctly.

[0164] For example, suppose you are developing a system that needs to extract order information from an order table in a database and convert this information into XML format according to certain rules for use by an external system. In this scenario, you first design a conversion rule listener module that listens to changes in the order table in the database (such as the insertion of a new order) and triggers the corresponding conversion function to convert the order information into XML format. Then, store the converted XML file in the file system and send it to the external system in some way (such as FTP, HTTP API, etc.).

[0165] In a specific implementation scheme, the defined conversion rules include the structure of the source data, the format of the target data, and specific steps on how to extract information from the source data and convert it into the target data.

[0166] In a specific implementation scheme, the computer language program is executed using a conversion encapsulation module, and the steps of saving information and triggering associated functions are finally completed, specifically:

[0167] 1. Choose the appropriate conversion package module

[0168] According to the requirements of the program and the target platform, select the appropriate conversion and packaging module. The conversion and packaging module can be a specialized software tool, library or framework used to convert the program from one form to another and encapsulate it into a package or component that can run on the target platform.

[0169] 2. Use the conversion package module to convert the program

[0170] Input the written program into the conversion and packaging module, follow the module's instructions or interface instructions to convert the program into a form suitable for execution on the target platform. During the conversion process, the program needs to be modified or optimized to ensure that it can run correctly on the target platform.

[0171] 3. Packaging Program

[0172] The converted program is packaged to generate an executable package or component. The packaging process includes adding necessary metadata, configuration files, dependent libraries, etc. to ensure that the program can run independently on the target platform.

[0173] 4. Execute the packaged program

[0174] Run the packaged program on the target platform. During execution, the program will process data, calculate results, and generate new data or files according to the predetermined logic.

[0175] 5. Information preservation and related function triggering

[0176] Information preservation: The data or results generated during program execution need to be properly preserved. This can be achieved by writing the data into files, databases, or cloud storage. When saving, attention should be paid to the format, security, and accessibility of the data.

[0177] Triggering of associated functions: If the program contains parts that need to trigger associated functions, such as updating records in the database, sending notifications, or triggering other program executions based on specific conditions, you need to set the corresponding logic in the program. When the program executes these logics, the corresponding associated functions will be automatically triggered.

[0178] 6. Verification and testing

[0179] Finally, the packaged program is verified and tested to ensure that it can correctly perform the expected functions and that the information preservation and association functions can be triggered as expected. Different operating environments and conditions need to be simulated during the verification and testing process to ensure the robustness and reliability of the program.

[0180] It should be noted that with the development of technology and the upgrading of tools, the execution method and steps of the conversion package module will change. In actual operation, it is necessary to respond flexibly and make adjustments according to specific circumstances.

[0181] Specifically, refer to Figure 3 and Figure 4 The DSL-based public dimension data conversion method is a method for quickly defining dimension data assets and their conversion relationships using business language, including the following steps: the business language script reading module reads the input script as a file stream, and then the file stream is input into the lexical parsing module, the lexical parsing module divides the characters in the stream into tokens according to predefined lexical rules, and inputs them into the syntax parser module; the syntax parser module constructs the token into a syntax tree according to predefined syntax rules; the syntax traversal module traverses all nodes of the tree and triggers a conversion rule listener at each node; the listener extracts information according to the defined conversion rules, assembles it into computer language program logic, executes the program, and finally completes the information preservation and associated function triggering.

[0182] In the above technical solution, the code template is abstracted through the componentized design principle to form a common macro. Each macro module is relatively independent and can be referenced in the code to reduce the amount of code. The business function is modeled through the configurable data conversion component of the DSL to form a model. When modeling, an existing model is introduced or a new model is created. The system performs model verification to avoid repeated modeling, which can simplify the bloated model. The parsed DSL is converted into metadata and stored in the database. The metadata can be used to build functions such as asset management, lineage display, indicator tracking, and attribution analysis.

[0183] exist Figure 2 In the embodiment of the present application, a public dimension data conversion system based on DSL is provided, comprising:

[0184] A business language script reading module, used for reading the input script as a file stream, and inputting the obtained file stream into the lexical parsing module;

[0185] A lexical analysis module, used for dividing the characters in the file stream into tokens, and inputting the obtained tokens into a syntax analyzer module;

[0186] A syntax parser module, used to construct the token into a syntax tree;

[0187] A grammar traversal module, used for traversing all nodes of the grammar tree and triggering a conversion rule listener module at each node;

[0188] A conversion rule listener module is used to extract information and assemble it into a computer language program;

[0189] The conversion and encapsulation module is used to execute the computer language program and finally complete the storage of information and the triggering of associated functions.

[0190] In the above technical solution, the business language script reading module is used to read the input script as a file stream, and the obtained file stream is input into the lexical parsing module; the lexical parsing module is used to segment the characters in the file stream into tokens, and the obtained tokens are input into the syntax parser module; the syntax parser module is used to construct the tokens into a syntax tree; the syntax traversal module is used to traverse all nodes of the syntax tree, and the conversion rule listener module is triggered at each node; the conversion rule listener module is used to extract information and assemble it into a computer language program; through the componentized design principle, the code template is abstracted to form a common macro, each macro module is relatively independent, and the macro can be referenced in the code to reduce the amount of code; the business function is modeled by the configurable data conversion component of the DSL to form a model, and an existing model is introduced or a new model is created during modeling. The system performs model verification to avoid repeated modeling, which can simplify the bloated model; the parsed DSL is formed into metadata, stored in the database, and the metadata is used to build functions such as asset management, lineage display, indicator tracking, and attribution analysis.

[0191] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.

[0192] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.

[0193] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0194] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.

Claims

1. A DSL-based public dimension data conversion method, characterized in that: The following steps are involved: Using the business language script reading module, the input script is read as a file stream, and the obtained file stream is input into the lexical analysis module; Using a lexical analysis module to segment characters in the file stream into tokens, and inputting the obtained tokens into a syntax analyzer module; Using a syntax parser module to construct the token into a syntax tree; Using a grammar traversal module, traverse all nodes of the grammar tree, and trigger a conversion rule listener module at each node; The information is extracted using the transformation rule listener module and assembled into a computer language program.

2. The DSL-based public dimension data conversion method according to claim 1, characterized in that: Also includes: The computer language program is executed using the conversion encapsulation module, and finally the information is saved and the associated function is triggered.

3. The DSL-based public dimension data conversion method according to claim 2, characterized in that: The steps of using the lexical analysis module to segment the characters in the file stream into tokens and inputting the obtained tokens into the syntax parser module are specifically as follows: The lexical analysis module is used to segment the characters in the file stream into tokens according to predefined lexical rules, and the obtained tokens are input into the syntax analyzer module.

4. The DSL-based public dimension data conversion method according to claim 3 is characterized in that: The predefined lexical rules include keywords and reserved words, identifiers, operators and delimiters, constants, comments, whitespace characters, string literals, character literals, and regular expressions.

5. The DSL-based public dimension data conversion method according to claim 4 is characterized in that: The steps of constructing the token into a syntax tree using a syntax parser module are as follows: The token is constructed into a syntax tree according to predefined syntax rules using a syntax parser module.

6. The DSL-based public dimension data conversion method according to claim 5, characterized in that: The predefined grammar rules include context-free grammar, form of rules, recursive rules, precedence and associativity, error handling, language-specific features, and example rules.

7. The DSL-based public dimension data conversion method according to claim 6, characterized in that: The steps of extracting information using the conversion rule listener module and assembling it into a computer language program are as follows: Using the transformation rule listener module, information is extracted according to the defined transformation rules and assembled into a computer language program.

8. The DSL-based public dimension data conversion method according to claim 7, characterized in that: The steps of extracting information according to the defined conversion rules and assembling it into a computer language program by using the conversion rule listener module specifically include: Define transformation rules; Design the transformation rule listener module; Build conversion logic; Configure the conversion rules and deploy the listener.

9. The DSL-based public dimension data conversion method according to claim 8, characterized in that: The defined conversion rules include the structure of source data, the format of target data, and specific steps on how to extract information from the source data and convert it into target data.

10. A DSL-based public dimension data conversion system, characterized in that: include: A business language script reading module, used for reading the input script as a file stream, and inputting the obtained file stream into the lexical parsing module; A lexical analysis module, used for dividing the characters in the file stream into tokens, and inputting the obtained tokens into a syntax analyzer module; A syntax parser module, used to construct the token into a syntax tree; A grammar traversal module, used for traversing all nodes of the grammar tree and triggering a conversion rule listener module at each node; A conversion rule listener module is used to extract information and assemble it into a computer language program; The conversion and encapsulation module is used to execute the computer language program and finally complete the storage of information and the triggering of associated functions.

Images