SAS language compilation method, device, equipment and readable storage medium

Abstract

The present invention discloses a SAS language compilation method, device, equipment and readable storage medium, wherein the SAS language compilation method comprises: receiving SAS code, inputting the SAS code into a preset preprocessing parser, outputting a first intermediate code, inputting the first intermediate code into a preset macro language parser to parse the macro language in the first intermediate code, outputting a second intermediate code, inputting the second intermediate code into a preset data and process step parser to parse the data step and process step in the second intermediate code, and outputting a target code. The method solves the technical problem of language conversion development and maintenance difficulties in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of language conversion technology of financial technology, and in particular to a SAS language compilation method, device, equipment and readable storage medium. Background Art

[0002] With the continuous development of financial technology, especially Internet technology finance, more and more technologies (such as distributed, blockchain, artificial intelligence, etc.) are applied in the financial field, but the financial industry also puts forward higher requirements on technology, such as higher requirements on the distribution of corresponding to-do items in the financial industry.

[0003] With the vigorous development of the software industry, there are more and more types of programming languages. In daily software programming development, it is often necessary to convert the source language into another language to realize the function of the source language. At present, in the prior art, the source language is usually analyzed through lexical syntax to generate intermediate code, and then the intermediate code is optimized in multiple layers to generate target code represented by machine code. However, the development workload of the language conversion method is large, resulting in slow development progress, which in turn makes language conversion development difficult. The machine code finally generated is difficult to read, which makes the maintenance of the machine code more difficult. Therefore, there are technical problems in the prior art that language conversion development and maintenance are difficult. Summary of the invention

[0004] The main purpose of the present invention is to provide a SAS language compilation method, device, equipment and readable storage medium, aiming to solve the technical problems of language conversion development and maintenance difficulties in the prior art.

[0005] To achieve the above object, an embodiment of the present invention provides a SAS language compilation method, which is applied to a SAS language compilation device, and includes:

[0006] Receive SAS code, input the SAS code into a preset preprocessing syntax analyzer, and output a first intermediate code;

[0007] Inputting the first intermediate code into a preset macro language syntax analyzer to parse the macro language in the first intermediate code and output a second intermediate code;

[0008] The second intermediate code is input into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output a target code.

[0009] Optionally, the preset macro language syntax analyzer includes a macro syntax analyzer, a macro variable value analyzer and a macro function call syntax analyzer.

[0010] The step of inputting the first intermediate code into a preset macro language parser to parse the macro language in the first intermediate code and outputting the second intermediate code comprises:

[0011] Inputting the first intermediate code into the macro syntax analyzer, parsing the macro syntax in the first intermediate code, and obtaining a first macro stage code;

[0012] Inputting the first macro phase code into the macro variable value analyzer, parsing the macro variables in the first macro phase code, and obtaining a second macro phase code;

[0013] The second macro stage code is input into a macro function call syntax analyzer, and the macro function in the second macro stage code is parsed to obtain the second intermediate code.

[0014] Optionally, the first intermediate code includes macro definitions and macro control statements.

[0015] The step of inputting the first intermediate code into the macro syntax analyzer, parsing the macro syntax in the first intermediate code, and obtaining the first macro stage code comprises:

[0016] Inputting the first intermediate code into the macro syntax analyzer to identify the macro definition and macro control statement based on a preset syntax pattern;

[0017] Performing language conversion on the macro definition and the macro control statement, generating Python language corresponding to the macro definition and the macro control statement, and obtaining macro syntax processing code;

[0018] The macro syntax processing code is spliced ​​with a preset fixed statement to obtain the first macro stage code.

[0019] Optionally, the step of inputting the second intermediate code into a preset data and process step parser to parse the data steps and process steps in the second intermediate code and outputting the target code includes:

[0020] Inputting the second intermediate code into a preset data and process step syntax analyzer to selectively filter comment statements in the second intermediate code;

[0021] Performing queue matching on the filtered second intermediate code to obtain one or more code queues;

[0022] Syntax parsing is performed on each of the code queues to obtain the target code.

[0023] Optionally, the preset data and process step syntax analyzer includes a second intermediate code lexical analyzer and a second intermediate code syntax analyzer,

[0024] The step of performing syntax analysis on each of the code queues to obtain the target code comprises:

[0025] The characters in the queue are converted into lexical symbols by the second intermediate code lexical analyzer, and each lexical symbol is sent to the second intermediate code syntax analyzer;

[0026] Recognize each of the lexical symbols by the second intermediate code syntax analyzer to obtain a syntax node;

[0027] Based on the syntax nodes, a syntax analysis tree is synthesized, and each syntax node of the syntax analysis tree is traversed to obtain the target code.

[0028] Optionally, the syntax node includes a data step node, and the second intermediate code includes a data step code.

[0029] The step of traversing each of the syntax nodes of the syntax analysis tree to obtain the target code comprises:

[0030] When traversing the data step nodes, based on the data step implementation process in the data step nodes, the data step code is simulated;

[0031] The target code is obtained by performing sentence pattern matching on the steps of the process simulation through a preset language runtime library.

[0032] Optionally, the syntax node includes a process step node, and the second intermediate code includes a process step code.

[0033] The step of traversing each of the syntax nodes of the syntax analysis tree to obtain the target code comprises:

[0034] When traversing the process step nodes, determining whether there are other data processing language statements in the process step code;

[0035] When the other data processing language statements exist in the process step code, calling the preset data management and analysis language syntax analyzer to perform syntax analysis on the other data processing language statements, and returning the data management and analysis language analysis result to the preset data and process step syntax analyzer;

[0036] The second intermediate code is parsed grammatically, and the target code is output based on the data management and analysis language parsing result.

[0037] The present invention further provides a SAS language compiling device, which is applied to a SAS language compiling device, and comprises:

[0038] A preprocessing module, configured to receive the SAS code, input the SAS code into a preset preprocessing syntax analyzer, and output a first intermediate code;

[0039] A macro phase module, used for inputting the first intermediate code into a preset macro language syntax analyzer to parse the macro language in the first intermediate code and output a second intermediate code;

[0040] The data and process step module is used to input the second intermediate code into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output the target code.

[0041] Optionally, the macro phase module includes:

[0042] A macro syntax submodule, used for inputting the first intermediate code into the macro syntax analyzer, parsing the macro syntax in the first intermediate code, and obtaining a first macro stage code;

[0043] A macro variable value submodule, configured to input the first macro phase code into the macro variable value analyzer, parse the macro variables in the first macro phase code, and obtain a second macro phase code;

[0044] The macro function calling submodule is used to input the second macro stage code into the macro function calling syntax analyzer, parse the macro function in the second macro stage code, and obtain the second intermediate code.

[0045] Optionally, the macro grammar submodule includes:

[0046] An input unit, configured to input the first intermediate code into the macro syntax analyzer to identify the macro definition and macro control statement based on a preset syntax pattern;

[0047] A macro syntax unit is used for performing language conversion on the macro definition and the macro control statement, generating Python language corresponding to the macro definition and the macro control statement, and obtaining a macro syntax processing code;

[0048] A splicing unit is used to splice the macro syntax processing code with a preset fixed statement to obtain the first macro stage code.

[0049] Optionally, the data and process step module includes:

[0050] A filtering submodule, used for inputting the second intermediate code into a preset data and process step syntax analyzer to selectively filter the comment statements in the second intermediate code;

[0051] A queue matching submodule, used for performing queue matching on the filtered second intermediate code to obtain one or more code queues;

[0052] The syntax parsing submodule is used to perform syntax parsing on each of the code queues to obtain the target code.

[0053] Optionally, the grammar parsing submodule includes:

[0054] A lexical analysis unit, configured to convert characters in the queue into lexical symbols through the second intermediate code lexical analyzer, and send each lexical symbol to the second intermediate code syntax analyzer;

[0055] A node identification unit, used for identifying each of the lexical symbols through the second intermediate code syntax analyzer to obtain a syntax node;

[0056] The traversal unit is used to synthesize a syntax analysis tree based on the syntax nodes, and traverse each syntax node of the syntax analysis tree to obtain the target code.

[0057] Optionally, the traversal unit includes:

[0058] A process simulation subunit, used for performing process simulation on the data step code based on the data step implementation process in the data step node when traversing the data step node;

[0059] The sentence pattern matching subunit is used to perform sentence pattern matching on the steps of the process simulation through a preset language runtime library to obtain the target code.

[0060] Optionally, the traversal unit includes:

[0061] A judgment subunit, used for judging whether there are other data processing language statements in the process step code when traversing the process step node;

[0062] A parsing subunit, for calling a preset data management and analysis language syntax analyzer to perform syntax analysis on the other data processing language statements when the other data processing language statements exist in the process step code, and returning the data management and analysis language parsing result to the preset data and process step syntax analyzer;

[0063] An output subunit is used to perform syntax analysis on the second intermediate code and output the target code based on the data management and analysis language analysis result.

[0064] The present invention also provides a SAS language compiling device, the SAS language compiling device comprising: a memory, a processor, and a program of the SAS language compiling method stored in the memory and executable on the processor, wherein the program of the SAS language compiling method can implement the steps of the SAS language compiling method as described above when executed by the processor.

[0065] The present invention also provides a readable storage medium, which is a computer-readable storage medium, and stores a program for implementing the SAS language compilation method. When the program of the SAS language compilation method is executed by a processor, the steps of the SAS language compilation method as described above are implemented.

[0066] The present application receives SAS code, inputs the SAS code into a preset preprocessing parser, outputs a first intermediate code, then inputs the first intermediate code into a preset macro language parser to parse the macro language in the first intermediate code, outputs a second intermediate code, and finally inputs the second intermediate code into a preset data and process step parser to parse the data step and process step in the second intermediate code, and outputs a target code. That is, the present application first receives the SAS code input by the user, inputs the SAS code into a preset preprocessing parser, obtains the first intermediate code, then inputs the first intermediate code into a preset macro language parser, obtains the second intermediate code, and finally inputs the second intermediate code into a preset data and process step parser to obtain the target code. That is, the present application only needs to perform syntax analysis on the macro language, data step and process step in the SAS code respectively through a preset preprocessing syntax analyzer, a preset macro language syntax analyzer and a preset data and process step syntax analyzer to obtain the target code. Therefore, when the SAS language compilation method of the present application performs language conversion, the development workload of the developer is small, and the target code is not limited to machine code and is easy to read and maintain. Therefore, the technical problem of difficult language conversion development and maintenance in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0069] Figure 1 This is a flow chart of the first embodiment of the SAS language compiling method of the present invention;

[0070] Figure 2 It is a schematic diagram of a syntax analysis tree of a syntax analyzer in a process step of the SAS language compilation method of the present invention;

[0071] Figure 3 This is a flow chart of a second embodiment of the SAS language compiling method of the present invention;

[0072] Figure 4 It is a schematic diagram of the data step implementation process in the SAS language compilation method of the present invention;

[0073] Figure 5 A flowchart of an implementation method of the second embodiment of the SAS language compilation method of the present invention combined with the first embodiment;

[0074] Figure 6 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0075] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0076] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0077] The present invention provides a SAS language compilation method, which is applied to a SAS language compilation device. In the first embodiment of the SAS language compilation method of the present application, referring to Figure 1 , the SAS language compilation method includes:

[0078] Step S10, receiving a SAS code, and inputting the SAS code into a preset preprocessing parser, and outputting a first intermediate code;

[0079] In this embodiment, it should be noted that the SAS language compilation method is implemented under the framework of the big data computing engine, and the preset preprocessing syntax analyzer includes a SAS code lexical analyzer and a SAS code syntax analyzer, wherein the SAS code lexical analyzer and the SAS code syntax analyzer are generated by using a plurality of customized syntax files and an automated language recognition tool ANTLR, and the SAS code is input into the preset preprocessing syntax analyzer in the form of a character string. The SAS language is a special data management and analysis language, and its data management function is similar to that of a database language, but it also adds many features of a general high-level programming language, such as conditional judgment, loops, etc., as well as functions and models dedicated to data analysis and statistics.

[0080] SAS code is received, and the SAS code is input into a preset preprocessing syntax analyzer, and a first intermediate code is output. Specifically, a SAS code input by a user is received, and the SAS code is input into a preset preprocessing syntax analyzer, so that characters in the SAS code are converted into SAS code lexical symbols through the SAS code lexical analyzer, and then the lexical symbols are identified as SAS code syntax nodes through the SAS code syntax analyzer, and a SAS code syntax analysis tree is synthesized, and then by traversing each syntax node in the SAS code syntax analysis tree, some ambiguous statements in the SAS code are filtered and processed to obtain the first intermediate code, wherein the filtered some ambiguous statements include block comment syntax / ** / , and the purpose of processing the some ambiguous statements is mainly to identify whether there is a semicolon after the macro function bracket, and if not, add a semicolon to unify the terminators of each statement, thereby reducing the parsing pressure of the subsequent syntax analyzer.

[0081] Step S20, inputting the first intermediate code into a preset macro language syntax analyzer to parse the macro language in the first intermediate code and output a second intermediate code;

[0082] In this embodiment, it should be noted that the preset macro language syntax analyzer is generated by customizing multiple syntax files using the automated language recognition tool ANTLR, and the preset macro language syntax analyzer includes a macro syntax analyzer, a macro variable value analyzer, and a macro function call syntax analyzer.

[0083] The first intermediate code is input into a preset macro language syntax analyzer to parse the macro language in the first intermediate code and output a second intermediate code. Specifically, the first intermediate code is first input into the macro syntax analyzer to identify and process the macro syntax to obtain a first macro stage code. The first macro stage code is then input into the macro variable value analyzer to parse the macro variables in the first macro stage code, extract the values ​​in the macro variables, and obtain a second macro stage code. Finally, the second macro stage code is input into the macro function call syntax analyzer to parse the function type and input parameters of the macro function in the second macro stage code, and run the function to obtain a second intermediate code.

[0084] The preset macro language parser includes a macro parser, a macro variable value parser and a macro function call parser.

[0085] The step of inputting the first intermediate code into a preset macro language parser to parse the macro language in the first intermediate code and outputting the second intermediate code comprises:

[0086] Step S21, inputting the first intermediate code into the macro syntax analyzer, parsing the macro syntax in the first intermediate code, and obtaining a first macro stage code;

[0087] In this embodiment, it should be noted that the macro parser includes a first intermediate code lexical analyzer and a first intermediate code syntax analyzer, and the first intermediate code is input into the macro parser in the form of a character string.

[0088] The first intermediate code is input into the macro syntax analyzer, and the macro syntax in the first intermediate code is parsed to obtain the first macro stage code. Specifically, the first intermediate code is input into the macro syntax analyzer, and the characters in the first intermediate code are converted into first intermediate code lexical symbols by the first intermediate code lexical analyzer, and then the first intermediate code lexical symbols are recognized as syntax nodes by the first intermediate code syntax analyzer, and then a first intermediate code syntax analysis tree is synthesized, and each syntax node in the first intermediate code syntax analysis tree is traversed to parse the macro syntax in the first intermediate code to obtain the first macro stage code.

[0089] Wherein, the first intermediate code includes macro definitions and macro control statements,

[0090] The step of inputting the first intermediate code into the macro syntax analyzer, parsing the macro syntax in the first intermediate code, and obtaining the first macro stage code comprises:

[0091] Step S211, inputting the first intermediate code into the macro syntax analyzer to identify the macro definition and macro control statement based on a preset syntax pattern;

[0092] In this embodiment, it should be noted that the preset grammatical patterns include macro definition and macro control statement grammatical patterns and other character grammatical patterns.

[0093] The first intermediate code is input into the macro syntax analyzer to identify the macro definition and macro control statement based on a preset syntax pattern. Specifically, the first intermediate code is input into the macro syntax analyzer to identify the macro definition and macro control statement based on the macro definition and macro control statement syntax pattern, and to identify and process other characters in text form based on the other character syntax patterns, wherein the macro definition and macro control statement include %LET statement, %IF statement, %ELSE statement, %DO statement, %PUT statement, %DO%TO statement, %DO%WHILE statement, %DO%UNTIL statement, macro function definition and data and process step syntax statements, etc.

[0094] Step S212, performing language conversion on the macro definition and the macro control statement, generating Python language corresponding to the macro definition and the macro control statement, and obtaining a macro syntax processing code;

[0095] In this embodiment, the macro definition and macro control statement are converted into a language, the Python language corresponding to the macro definition and macro control statement is generated, and the macro syntax processing code is obtained. Specifically, based on the correspondence between the macro definition and macro control statement and the Python language statement, the macro definition and macro control statement are converted into a language, and the Python language corresponding to the macro definition and macro control statement is generated. For example, the macro statement "%DO x=1%TO 3;" is converted into the Python language code of "for x inrange(1,4):", and the macro statement "%macro func();" is converted into the Python language code of "def func():", and then the macro syntax processing code is obtained, wherein the macro syntax processing code includes the characters corresponding to the Python language and other characters. For other characters, the interface of the next syntax analyzer will be called to wait for the next stage of processing.

[0096] Step S213, concatenating the macro syntax processing code with a preset fixed statement to obtain the first macro stage code.

[0097] In this embodiment, the macro syntax processing code is spliced ​​with preset fixed statements to obtain the first macro stage code. Specifically, the macro syntax processing code is spliced ​​with preset fixed statements, wherein the preset fixed statements include codes of third-party modules and codes corresponding to initialization objects, etc., to form a complete Python program. Furthermore, due to the characteristics of Python dynamic language, the exec function is applied to the generated statement, and the macro syntax analyzer is run to output the Python program to obtain the first macro stage code.

[0098] Step S22, inputting the first macro phase code into the macro variable value analyzer, parsing the macro variables in the first macro phase code, and obtaining a second macro phase code;

[0099] In this embodiment, it should be noted that the macro variable value analyzer includes a first macro stage code lexical analyzer and a first macro stage code syntax analyzer, and the first macro stage code is input into the macro syntax analyzer in the form of a character string, wherein, in this embodiment, when the characters of the previous syntax analyzer are passed to the next syntax analyzer, the global variables and local variables in the previous syntax analyzer are also passed over, and the global variables and local variables of the next syntax analyzer are merged together.

[0100] The first macro stage code is input into the macro variable value analyzer, and the macro variables in the first macro stage code are parsed to obtain the second macro stage code. Specifically, the first macro stage code is input into the macro variable value analyzer, so that the characters in the first macro stage code are converted into first macro stage code lexical symbols through the first macro stage code lexical analyzer, and then the first macro stage code lexical symbols are identified as syntax nodes through the first macro stage code syntax analyzer, and then the first macro stage code syntax analysis tree is synthesized. Further, each syntax node in the first macro stage code syntax analysis tree is traversed to determine whether the macro variable is defined in the outer layer. When the macro variable is defined in the outer layer, the macro variable is a global variable, and the global variable is updated and returned. When the macro variable is not defined in the outer layer, the macro variable is a local variable, and the local variable will not be saved. That is, the macro variable in the first macro stage code is parsed to obtain the second macro stage code.

[0101] Step S23, inputting the second macro stage code into a macro function call syntax analyzer, parsing the macro function in the second macro stage code, and obtaining the second intermediate code.

[0102] In this embodiment, it should be noted that the macro function call syntax analyzer includes a second macro stage code lexical analyzer and a second macro stage code syntax analyzer, and the second macro stage code is input into the macro function call syntax analyzer in the form of a character string. The function of the macro function call syntax analyzer is to parse the function type and input parameters of the macro function and run the function. The second macro stage code corresponds to a second macro stage code syntax analysis tree.

[0103] The second macro stage code is input into a macro function call syntax analyzer, and the macro function in the second macro stage code is parsed to obtain the second intermediate code. Specifically, the second macro stage code is input into a macro function call syntax analyzer, and the function type incoming parameters of the macro function are parsed by traversing each syntax node of the second macro stage code syntax analysis tree, wherein the parsing of the incoming parameters supports four operations between constants and macro variables, and between constants and variables. The parsing of the function type is to classify the system function with the function name %EVAL and convert it into the corresponding eval function of Python for running, that is, the macro function in the second macro stage code is parsed to obtain the second intermediate code.

[0104] Step S30, inputting the second intermediate code into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output the target code.

[0105] In this embodiment, it should be noted that the preset data and process step syntax analyzer includes a second intermediate code lexical analyzer and a second intermediate code syntax analyzer, and the preset data and process step syntax analyzer is generated by using a plurality of custom syntax files and an automated language recognition tool ANTLR tool, such as Figure 2 As shown, the second intermediate code corresponds to a second intermediate code syntax analysis tree, that is, the preset data and process step syntax analyzer corresponds to a data and process step syntax analyzer syntax analysis tree, wherein parse is the root node, dataProcStmt represents multiple code blocks of the same structure, L1, L2 and L3 are tree layers where leaf nodes of the syntax analysis tree are located, wherein the leaf nodes include dataStep nodes, setStmt nodes and ifStmt nodes, etc., each code block represents a data step or a process step, and the target code refers to other language codes other than SAS code.

[0106] The second intermediate code is input into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output the target code. Specifically, the second intermediate code is input into a preset data and process step syntax analyzer, and each node of the second intermediate code syntax analysis tree is traversed in depth-first order through the root node. Further, each node will traverse the child nodes until the leaf node, and return the target code, that is, the data steps and process steps in the second intermediate code are parsed and the target code is output.

[0107] This embodiment receives SAS code, inputs the SAS code into a preset preprocessing parser, outputs a first intermediate code, then inputs the first intermediate code into a preset macro language parser to parse the macro language in the first intermediate code, outputs a second intermediate code, and finally inputs the second intermediate code into a preset data and process step parser to parse the data step and process step in the second intermediate code, and outputs a target code. That is, this embodiment first receives the SAS code input by the user, inputs the SAS code into a preset preprocessing parser, obtains the first intermediate code, then inputs the first intermediate code into a preset macro language parser, obtains the second intermediate code, and finally inputs the second intermediate code into a preset data and process step parser to obtain the target code. That is, the present embodiment only needs to perform syntax analysis on the macro language, data step and procedure step in the SAS code respectively through the preset preprocessing syntax analyzer, the preset macro language syntax analyzer and the preset data and procedure step syntax analyzer, so as to obtain the target code. Therefore, when the SAS language compilation method of the present embodiment performs language conversion, the development workload of the developer is small, and the target code is not limited to the machine code and is easy to read and maintain. Therefore, the technical problem of the difficulty of language conversion development and maintenance in the prior art is solved.

[0108] Further, refer to Figure 3 Based on the first embodiment of the present application, in another embodiment of the SAS language compilation method, the step of inputting the second intermediate code into a preset data and process step parser to parse the data steps and process steps in the second intermediate code and outputting the target code includes:

[0109] Step S31, inputting the second intermediate code into a preset data and process step syntax analyzer to selectively filter the comment statements in the second intermediate code;

[0110] In this embodiment, the second intermediate code is input into the preset data and process step syntax analyzer to selectively filter the comment statements in the second intermediate code. Specifically, the second intermediate code is input into the preset data and process step syntax analyzer, and the second intermediate code is detected by the second intermediate code lexical analyzer to determine whether there are comment statements in the second intermediate code. If so, the comment statements are filtered. One of the methods for determining whether there are comment statements in the second intermediate code is to establish a queue with a length of 3, and add each character flowing through the queue to the queue, wherein the old character will be replaced by the new character. If one of the following three conditions is met, it is a comment statement: (a) when the character just entering the queue is "*", the queue is 1 or 2; (b) the queue length is 3, and the first or second character before the "*" in the queue is ";"; (c) when the queue length is 3, and the first character before the "*" in the queue is "\r\n\t", and the second character before it is "*".

[0111] Step S32, performing queue matching on the filtered second intermediate code to obtain one or more code queues;

[0112] In this embodiment, the second intermediate code after filtering is queue matched to obtain one or more code queues. Specifically, the second intermediate code after filtering is detected by the second intermediate code syntax analyzer to determine whether the run / quit lexical symbol is matched. When the run / quit lexical symbol is matched, the run / quit lexical symbol is used as the code block end mark. At this time, all characters from the previous end mark to the current end mark are used as a code block. If there is no previous end mark, all characters from the first character of the second intermediate code to the current end mark are used as a code block, wherein the code block includes the first character, that is, a queue is obtained. When the run / quit lexical symbol is not matched, the matching continues until the run / quit lexical symbol is matched.

[0113] Step S33, performing syntax analysis on each of the code queues to obtain the target code.

[0114] In this embodiment, syntax parsing is performed on each of the code queues to obtain the target code. Specifically, the characters in each of the queues are taken out, and the second intermediate code syntax analysis tree is established, wherein one queue corresponds to a code block, and syntax parsing is performed on each of the code queues by traversing each node in the second intermediate code syntax analysis tree to obtain the target code.

[0115] The preset data and process step syntax analyzer includes a second intermediate code lexical analyzer and a second intermediate code syntax analyzer.

[0116] The step of performing syntax analysis on each of the code queues to obtain the target code comprises:

[0117] Step S331, converting the characters in the queue into lexical symbols through the second intermediate code lexical analyzer, and sending each lexical symbol to the second intermediate code syntax analyzer;

[0118] In this embodiment, the characters in the queue are converted into lexical symbols by the second intermediate code lexical analyzer, and each lexical symbol is sent to the second intermediate code syntax analyzer. Specifically, the characters in the queue are combined by the second intermediate code lexical analyzer, and then the characters are combined into valid words, that is, converted into lexical symbols, and further, each lexical symbol is sent to the second intermediate code syntax analyzer.

[0119] Step S332, identifying each of the lexical symbols through the second intermediate code syntax analyzer to obtain a syntax node;

[0120] In this embodiment, each lexical symbol is identified by the second intermediate code syntax analyzer to obtain a syntax node. Specifically, each lexical symbol is subjected to syntax analysis by the second intermediate code syntax analyzer, that is, each lexical symbol is taken as input to analyze whether the code composed of each lexical symbol complies with preset syntax rules. Furthermore, if it complies with the preset syntax rules, the semantics of the code composed of each lexical symbol under the preset syntax rules is used to obtain an intermediate code, that is, a syntax node.

[0121] Step S333: synthesize a syntax analysis tree based on the syntax nodes, and traverse each syntax node of the syntax analysis tree to obtain the target code.

[0122] In this embodiment, based on the syntax nodes, a syntax parse tree is synthesized, and each syntax node of the syntax parse tree is traversed to obtain the target code. Specifically, based on the syntax nodes, a syntax parse tree corresponding to the second intermediate code is synthesized, and each syntax node of the syntax parse tree is traversed, that is, the intermediate code is optimized to obtain the target code.

[0123] wherein the syntax node comprises a data step node, the second intermediate code comprises a data step code,

[0124] In step S333, the step of traversing each syntax node of the syntax analysis tree to obtain the target code includes:

[0125] A10, when traversing the data step node, based on the data step implementation process in the data step node, the data step code is simulated;

[0126] In this embodiment, it should be noted that the target code includes Python language code, C language code, Java language code and other non-SAS language codes. In the SAS language, the data step corresponds to some data processing steps of the data processing method, and the steps in the data step implementation process correspond to the step processes required by other languages ​​to implement the data processing steps. For example, Figure 4The figure is a flowchart of the steps required for Python language to implement the data processing steps, that is, a flowchart of the data steps required for Python language to implement the data processing steps, wherein the steps in the data step implementation flowchart include: (1) introducing or creating a data table, which corresponds to the set / datalines statement of SAS; (2) conditional screening of data, corresponding to the keep / drop / rename / where keywords in the set sentence pattern; (3) inserting new variables, wherein the new variables are obtained from the syntax analysis process, the purpose is to set default values ​​for the new variables, and it can support the situation where there is only an if statement but no else statement to assign variables; (4) automatically selecting distributed or stand-alone mode according to the supported syntax sentence pattern, wherein a small number of those that have strict requirements on the row order will adopt the stand-alone mode, such as retain, set end, etc.; (5) The dotted box represents a custom iterator, which will act on an rdd.mapPartitions operation. The iterator mainly processes the data of each row. Each round of operation is like simulating the row-by-row operation of SAS, but in essence, Spark uses a parallel working mechanism, unlike SAS which can only execute sequentially. In each round of the iterator, if there is a retain statement, the value of the retained variable will be assigned as the initial value of the variable in the current row; (6) Convert null value data to infinitely small, which is consistent with SAS; (7) Process common statements, which include data processing statements such as conditional control, arithmetic operations, loops, character processing, etc. Most of them can be found in the runtime library. Templates for use; (8) Update and Store the value of the retained variable; (9) Infinitely small data is converted to null values ​​to match the storage format of the database in Spark; (10) Determine whether to output the current row of data, corresponding to the output statement, and output each row in the default data table; (11) Converting rdd to dataframe requires determining the data type of each field. In order to support fields with multiple null values ​​at the beginning of the table, the default solution is to use the data type of the first 100 rows and merge it with the original field type of the referenced dataframe; (12) Filter the data again, corresponding to the keep / drop / rename / where keywords of the data sentence; (13) Save the data to HIVE or a temporary view. For statements that explicitly specify the output table name, such as "data testdb.t1;", the data will be kept in HIVE. If the table name is not specified, the data will be stored in a temporary view.

[0127] Specifically, when the data step nodes are traversed, the data step code is simulated based on the data step implementation process in the data step nodes. Specifically, when the data step nodes are traversed, the data step code is simulated based on the python language based on the data step implementation process in the data step nodes, so that all functions and data processing processes that can be realized by the SAS code corresponding to the data step code can be realized using the python language.

[0128] A20, performing sentence pattern matching on the steps of the process simulation through a preset language runtime library to obtain the target code.

[0129] In this embodiment, it should be noted that the preset language runtime library includes a python runtime library, a java runtime library, etc., wherein, for example, the python runtime library uses python / Spark to encapsulate SAS name patterns similar to static function usage, and when converted into target code, the matching sentence patterns can be replaced by the python runtime library.

[0130] The target code is obtained by matching the sentence patterns of the steps of the process simulation through the preset language runtime library. Specifically, the sentence patterns of the steps of the process simulation are matched through the preset language runtime library, and each step of the process simulation is expressed using the sentence patterns in the preset language runtime library, thereby obtaining the queue code corresponding to each data step. In this embodiment, dataframe is used as a temporary storage object for data, and a temporary dataframe is output at the end of each data step or process step. For statements with clearly specified output table names, such as "datatestdb.t1;", the data will be written to the testdb library of HIVE. If the table name is not specified, the data will be stored as a temporary view, and the codes corresponding to all queues will be output eventually to obtain the target code.

[0131] The syntax node includes a process step node, the second intermediate code includes a process step code,

[0132] In step S333, the step of traversing each syntax node of the syntax analysis tree to obtain the target code includes:

[0133] B10, when traversing the process step nodes, determining whether there are other data processing language statements in the process step code;

[0134] In this embodiment, when the process step node is traversed, it is determined whether there are other data processing language statements in the process step code, wherein the other data processing language statements include all types of data management and analysis languages ​​​​except SAS language. Specifically, when the process step node is traversed, the preset data and the process step syntax analyzer are used to detect whether there are other data processing language statements in the process step node, that is, it is determined whether there are other data processing language statements in the process step code.

[0135] B20, when the other data processing language statements exist in the process step code, calling the preset data management and analysis language syntax analyzer to perform syntax analysis on the other data processing language statements, and returning the data management and analysis language analysis result to the preset data and process step syntax analyzer;

[0136] In this embodiment, when the other data processing language statements exist in the process step code, the preset data management and analysis language syntax analyzer is called to perform syntax analysis on the other data processing language statements, and the data management and analysis language analysis results are returned to the preset data and process step syntax analyzer. Specifically, when the other data processing language statements exist in the process step code, the preset data management and analysis language syntax analyzer is called to perform syntax analysis on the other data processing language statements, wherein the preset data management and analysis language syntax analyzer will match all other data processing language statements, and will perform special processing on the select clause represented by multiple lines. Specifically, the other data processing language statements will first pass through the special lexical analysis in the preset data management and analysis language syntax analyzer. The analyzer performs lexical analysis on the other data processing language statements to obtain data management and analysis language lexical symbols corresponding to the other data processing language statements, wherein the lexical symbols include ignored blank characters and carriage returns, etc., and then replaces multiple consecutive blanks and carriage returns with a blank character, so that other data processing language statements represented by multiple lines can be converted into single-line other data processing language statements, and in this process, if a function name in the other data processing language statement is detected, the function name is converted into a corresponding function name in the SPARK data management and analysis language, for example, the function name "mean" is replaced with "avg", and then the data management and analysis language parsing result of the other data processing language statement is obtained, and the data management and analysis language parsing result is returned to the preset data and process step syntax analyzer.

[0137] B30, perform syntax analysis on the second intermediate code, and output the target code based on the data management and analysis language analysis result.

[0138] In this embodiment, the second intermediate code is parsed, and the target code is output based on the data management and analysis language parsing result. Specifically, the remaining statements in the second intermediate code except the other data processing language statements are parsed, that is, by traversing the process step nodes in the syntax analysis tree of the second intermediate code, the remaining statements are optimized. Further, in combination with the data management and analysis language parsing result, the target code is output, such as Figure 5 The figure shows a schematic diagram of an implementation method of the present embodiment combined with the first embodiment, wherein the SAS code is compiled into Python code, the Pro SQL parser is a parser in the preset data management and analysis language parser, a.g4, b.g4, c.g4, d.g4 and e.g4 in the figure all represent custom grammar files, based on the custom grammar file, the lexical analyzer and grammar analyzer corresponding to the custom grammar file can be generated by using the automated language recognition tool ANTLR tool.

[0139] This embodiment selectively filters the comment statements in the second intermediate code by inputting the second intermediate code into the preset data and the process step syntax analyzer, and then performs queue matching on the filtered second intermediate code to obtain one or more code queues, and finally performs syntax analysis on each of the code queues to obtain the target code. That is, this embodiment inputs the second intermediate code into the preset data and the process step syntax analyzer, first selectively filters the second intermediate code, then obtains the queue, and finally performs syntax analysis on the queue to obtain the target code. That is, this implementation can realize the conversion of the second intermediate code into the target code, and the method only needs to input the second intermediate code into the preset data and the process step syntax analyzer to output the target code, and the target code is not limited to machine code, so it lays a foundation for solving the technical problems of language conversion development and maintenance difficulties in the prior art.

[0140] Reference Figure 6 , Figure 6 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0141] like Figure 6As shown, the SAS language compilation device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0142] Optionally, the SAS language compilation device may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. The rectangular user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0143] Those skilled in the art will understand that Figure 6 The SAS language compiling device structure shown in the figure does not constitute a limitation on the SAS language compiling device, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0144] like Figure 6 As shown, the memory 1005 as a computer storage readable storage medium may include an operating system, a network communication module and a SAS language compiler. The operating system is a program that manages and controls the hardware and software resources of the SAS language compiler device, and supports the operation of the SAS language compiler and other software and / or programs. The network communication module is used to realize the communication between the components inside the memory 1005, and the communication with other hardware and software in the SAS language compiler system.

[0145] exist Figure 6 In the SAS language compiling device shown, the processor 1001 is used to execute the SAS language compiling program stored in the memory 1005 to implement the steps of any one of the above-mentioned SAS language compiling methods.

[0146] The specific implementation of the SAS language compiling device of the present invention is basically the same as the above-mentioned SAS language compiling method embodiments, and will not be repeated here.

[0147] The present invention also provides a SAS language compiling device, the SAS language compiling device comprising:

[0148] A preprocessing module, configured to receive the SAS code, input the SAS code into a preset preprocessing syntax analyzer, and output a first intermediate code;

[0149] A macro phase module, used for inputting the first intermediate code into a preset macro language syntax analyzer to parse the macro language in the first intermediate code and output a second intermediate code;

[0150] The data and process step module is used to input the second intermediate code into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output the target code.

[0151] Optionally, the macro phase module includes:

[0152] A macro syntax submodule, used for inputting the first intermediate code into the macro syntax analyzer, parsing the macro syntax in the first intermediate code, and obtaining a first macro stage code;

[0153] A macro variable value submodule, configured to input the first macro phase code into the macro variable value analyzer, parse the macro variables in the first macro phase code, and obtain a second macro phase code;

[0154] The macro function calling submodule is used to input the second macro stage code into the macro function calling syntax analyzer, parse the macro function in the second macro stage code, and obtain the second intermediate code.

[0155] Optionally, the macro grammar submodule includes:

[0156] An input unit, configured to input the first intermediate code into the macro syntax analyzer to identify the macro definition and macro control statement based on a preset syntax pattern;

[0157] A macro syntax unit is used for performing language conversion on the macro definition and the macro control statement, generating Python language corresponding to the macro definition and the macro control statement, and obtaining a macro syntax processing code;

[0158] A splicing unit is used to splice the macro syntax processing code with a preset fixed statement to obtain the first macro stage code.

[0159] Optionally, the data and process step module includes:

[0160] A filtering submodule, used for inputting the second intermediate code into a preset data and process step syntax analyzer to selectively filter the comment statements in the second intermediate code;

[0161] A queue matching submodule, used for performing queue matching on the filtered second intermediate code to obtain one or more code queues;

[0162] The syntax parsing submodule is used to perform syntax parsing on each of the code queues to obtain the target code.

[0163] Optionally, the grammar parsing submodule includes:

[0164] A lexical analysis unit, configured to convert characters in the queue into lexical symbols through the second intermediate code lexical analyzer, and send each lexical symbol to the second intermediate code syntax analyzer;

[0165] A node identification unit, which obtains a syntax node by identifying each of the lexical symbols through the second intermediate code syntax analyzer;

[0166] The traversal unit is used to synthesize a syntax analysis tree based on the syntax nodes, and traverse each syntax node of the syntax analysis tree to obtain the target code.

[0167] Optionally, the traversal unit includes:

[0168] A process simulation subunit, used for performing process simulation on the data step code based on the data step implementation process in the data step node when traversing the data step node;

[0169] The sentence pattern matching subunit is used to perform sentence pattern matching on the steps of the process simulation through a preset language runtime library to obtain the target code.

[0170] Optionally, the traversal unit includes:

[0171] A judgment subunit, used for judging whether there are other data processing language statements in the process step code when traversing the process step node;

[0172] A parsing subunit, for calling a preset data management and analysis language syntax analyzer to perform syntax analysis on the other data processing language statements when the other data processing language statements exist in the process step code, and returning the data management and analysis language parsing result to the preset data and process step syntax analyzer;

[0173] An output subunit is used to perform syntax analysis on the second intermediate code and output the target code based on the data management and analysis language analysis result.

[0174] The specific implementation of the SAS language compiling device of the present invention is basically the same as the above-mentioned SAS language compiling method embodiments, and will not be described in detail here.

[0175] The present invention provides a readable storage medium, which is a computer-readable storage medium, and the readable storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to implement the steps of any one of the above-mentioned SAS language compilation methods.

[0176] The specific implementation of the readable storage medium of the present invention is basically the same as the above-mentioned SAS language compilation method embodiments, and will not be repeated here.

[0177] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent scope of the present invention.

Claims

1. A SAS language compilation method, characterized in that: The SAS language compilation method comprises: Receive SAS code, input the SAS code into a preset preprocessing syntax analyzer, and output a first intermediate code; Inputting the first intermediate code into a preset macro language syntax analyzer to parse the macro language in the first intermediate code and output a second intermediate code; The preset macro language parser includes a macro parser, a macro variable value parser and a macro function call parser, the first intermediate code includes macro definitions and macro control statements, The step of inputting the first intermediate code into a preset macro language parser to parse the macro language in the first intermediate code and outputting the second intermediate code comprises: Inputting the first intermediate code into the macro syntax analyzer to identify the macro definition and macro control statement based on a preset syntax pattern; Performing language conversion on the macro definition and the macro control statement, generating Python language corresponding to the macro definition and the macro control statement, and obtaining macro syntax processing code; The macro syntax processing code is spliced ​​with a preset fixed statement to obtain a first macro stage code; Inputting the first macro stage code into the macro variable value analyzer, so as to convert characters in the first macro stage code into first macro stage code lexical symbols through the first macro stage code lexical analyzer; Recognize the first macro-stage code lexical symbols into syntax nodes by the first macro-stage code syntax analyzer, and synthesize a first macro-stage code syntax analysis tree; Traversing each syntax node in the syntax analysis tree of the first macro stage code to determine whether the macro variable is defined in the outer layer; If the macro variable is defined in the outer layer, determining that the macro variable is a global variable, updating and returning the global variable; If the macro variable is not defined in the outer layer, the macro variable is determined to be a local variable, and the macro variable in the first macro phase code is parsed to obtain a second macro phase code; Inputting the second macro stage code into a macro function call syntax analyzer, parsing the macro function in the second macro stage code, and obtaining the second intermediate code; The second intermediate code is input into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output a target code.

2. The SAS language compilation method according to claim 1, characterized in that: The step of inputting the second intermediate code into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output the target code comprises: Inputting the second intermediate code into a preset data and process step syntax analyzer to selectively filter comment statements in the second intermediate code; Performing queue matching on the filtered second intermediate code to obtain one or more code queues; Syntax parsing is performed on each of the code queues to obtain the target code.

3. The SAS language compilation method according to claim 2, characterized in that: The preset data and process step syntax analyzer includes a second intermediate code lexical analyzer and a second intermediate code syntax analyzer, The step of performing syntax analysis on each of the code queues to obtain the target code comprises: Convert the characters in the queue into lexical symbols by the second intermediate code lexical analyzer, and send each lexical symbol to the second intermediate code syntax analyzer; Recognize each of the lexical symbols by the second intermediate code syntax analyzer to obtain a syntax node; Based on the syntax nodes, a syntax analysis tree is synthesized, and each syntax node of the syntax analysis tree is traversed to obtain the target code.

4. The SAS language compilation method according to claim 3, characterized in that: The syntax node includes a data step node, the second intermediate code includes a data step code, The step of traversing each grammar node of the grammar analysis tree to obtain the target code comprises: When traversing the data step nodes, based on the data step implementation process in the data step nodes, the data step code is simulated; The target code is obtained by performing sentence pattern matching on the steps of the process simulation through a preset language runtime library.

5. The SAS language compiling method as claimed in claim 3, characterized in that: The syntax node includes a process step node, the second intermediate code includes a process step code, The step of traversing each grammar node of the grammar analysis tree to obtain the target code comprises: When traversing the process step nodes, determining whether there are other data processing language statements in the process step code; When the other data processing language statements exist in the process step code, calling the preset data management and analysis language syntax analyzer to perform syntax analysis on the other data processing language statements, and returning the data management and analysis language analysis result to the preset data and process step syntax analyzer; The second intermediate code is parsed grammatically, and the target code is output based on the data management and analysis language parsing result.

6. A SAS language compiling device, characterized in that: The SAS language compiling device is applied to a SAS language compiling device, and the SAS language compiling device comprises: A preprocessing module, used for receiving SAS code, inputting the SAS code into a preset preprocessing parser, and outputting a first intermediate code, wherein the preset macro language parser includes a macro parser, a macro variable value parser, and a macro function call parser, and the first intermediate code includes a macro definition and a macro control statement; A macro phase module is used for inputting the first intermediate code into a preset macro language syntax analyzer to parse the macro language in the first intermediate code and output a second intermediate code, wherein the macro phase module is specifically used for inputting the first intermediate code into the macro syntax analyzer to identify the macro definition and macro control statement based on a preset syntax pattern; performing language conversion on the macro definition and macro control statement to generate a Python language corresponding to the macro definition and macro control statement to obtain a macro syntax processing code; splicing the macro syntax processing code with a preset fixed statement to obtain a first macro phase code; inputting the first macro phase code into the macro variable value analyzer to convert the first macro phase code into a second intermediate code through the first macro phase code lexical analyzer. The characters are converted into lexical symbols of the first macro stage code; the lexical symbols of the first macro stage code are identified as grammatical nodes by the first macro stage code syntax analyzer, and a first macro stage code syntax analysis tree is synthesized; each syntax node in the first macro stage code syntax analysis tree is traversed to determine whether the macro variable is defined in the outer layer; if the macro variable is defined in the outer layer, the macro variable is determined to be a global variable, and the global variable is updated and returned; if the macro variable is not defined in the outer layer, the macro variable is determined to be a local variable, and the macro variable in the first macro stage code is parsed to obtain the second macro stage code; the second macro stage code is input into the macro function call syntax analyzer, and the macro function in the second macro stage code is parsed to obtain the second intermediate code; The data and process step module is used to input the second intermediate code into a preset data and process step syntax analyzer to parse the data steps and process steps in the second intermediate code and output the target code.

7. A SAS language compiling device, characterized in that: The SAS language compiling device comprises: a memory, a processor, and a program stored in the memory for implementing the SAS language compiling method. The memory is used to store a program for implementing a SAS language compilation method; The processor is used to execute a program for implementing the SAS language compilation method to implement the steps of the SAS language compilation method according to any one of claims 1 to 5.

8. A readable storage medium, characterized in that: The readable storage medium stores a program for implementing the SAS language compilation method, and the program for implementing the SAS language compilation method is executed by a processor to implement the steps of the SAS language compilation method according to any one of claims 1 to 5.

Images