Method, device and system for syntax analysis in data processing system

Abstract

This application discloses a method, apparatus, and system for grammatical analysis in a data processing system. The method comprises: performing grammatical analysis on a statement to be executed; and during the grammatical analysis process, invoking a corresponding operator processing program by setting a callback interface for the generated operator operation. Using the grammatical analysis method in a data processing system described in this application, the operator processing program that processes operator operations based on an abstract syntax tree in the data processing system can be pre-positioned to the abstract syntax tree generation stage, avoiding the generation of a large number of abstract syntax tree intermediate objects, thereby optimizing the processing efficiency of the data processing system and reducing system resource usage.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and more particularly to a method, device, and system for grammatical analysis in a data processing system. Furthermore, the present application also relates to a method and device for grammatical analysis of SQL statements in a data storage system. Background Art

[0002] In recent years, with the rapid development of network technology, the development requirements for data processing systems have also changed rapidly. At the same time, the SQL statements that data processing systems need to execute have become increasingly complex. Improving the efficiency of SQL statement parsing and optimizing the resource utilization of data processing systems have become common technical goals pursued by those skilled in the art.

[0003] To address the aforementioned technical issues, the analysis process for executing statements is typically divided into two phases: the first phase is the syntax analysis phase, in which an abstract syntax tree is generated based on the statement to be executed; the second phase is the abstract syntax tree processing phase, in which processing logic is executed on each node of the generated abstract syntax tree. SQL (Structured Query Language), abbreviated as SQL, is a special-purpose programming language and a database query and programming language used to access data, as well as query, update, and manage relational data processing systems. It does not require users to specify data storage methods, nor does it require users to understand specific data storage methods. Therefore, different data processing systems with completely different underlying structures can use the same structured query language as an interface for data input and management.

[0004] However, as users send more and more execution statements to the data processing system through applications, the existing method of generating an abstract syntax tree and then analyzing and processing it usually produces a large number of intermediate objects, resulting in high system resource usage, affecting the efficiency of syntax analysis in the data processing system, and thus failing to meet the actual needs of users. Summary of the Invention

[0005] The present application provides a method, device and system for grammatical analysis in a data processing system to solve the problem of low efficiency of grammatical analysis in data processing systems in the prior art. In addition, the present application also provides a method and device for grammatical analysis of SQL statements in a data storage system.

[0006] The method of grammatical analysis in a data processing system provided in the present application includes: performing grammatical analysis on statements to be executed; during the grammatical analysis process, calling the corresponding operator processing program by setting a callback interface for the generated operator operation.

[0007] Optionally, the data processing system further includes: determining whether the syntax analysis is completed and whether the called operator processing program is executed; if both of the above determinations are yes, generating a syntax analysis result that has brought back the processing result of the operator processing program.

[0008] Optionally, the data processing system includes a data storage system, and accordingly, the statements to be executed include SQL statements.

[0009] Optionally, during the syntax analysis process, the corresponding operator handler is called by setting a callback interface for the generated operator operation, including: in the execution plan formation stage during the syntax analysis of the SQL statement to be executed in the data storage system, the corresponding operator handler is called by setting a callback interface for the operator operation of the generated SQL statement, wherein the execution plan is an abstract syntax tree generated by the data storage system based on the SQL statement and composed of several operator operations in a set logical order.

[0010] Optionally, the operator operation is a basic operation program generated by analyzing the SQL statement in the data storage system.

[0011] Optionally, the operator processing program is used to perform preprocessing operations on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement in the data storage system; wherein, the preprocessing operation is a processing operation that needs to be performed by the processing module after generating the execution plan.

[0012] Optionally, before executing the step of performing a pre-processing operation on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement, the following step is executed: performing a legality check on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement; the legality check is to determine whether the data structure of the operator operation of the SQL statement complies with predetermined SQL grammar rules.

[0013] Optionally, the syntax analysis result includes an operator operation processing result generated after the corresponding operator processing program has been called through the callback interface, as well as an operator operation without calling the callback interface.

[0014] Correspondingly, the present application also provides a device for syntax analysis in a data processing system, including: a first syntax analysis unit and a first calling unit; the first syntax analysis unit is used to perform syntax analysis on statements to be executed; the first calling unit is used to call the corresponding operator processing program during the syntax analysis process by setting a callback interface for the generated operator operation.

[0015] Optionally, the data processing system further includes: a first judgment unit, configured to judge whether the syntax analysis is completed and whether the called operator processing program is executed; if the above judgments are both yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated.

[0016] Optionally, the data processing system includes a data storage system, and accordingly, the statements to be executed include SQL statements.

[0017] Optionally, the first calling unit is specifically used to: in the execution plan formation stage during the syntax analysis of the SQL statement to be executed in the data storage system, call the corresponding operator processing program by setting a callback interface for the operator operation of the generated SQL statement, wherein the execution plan is an abstract syntax tree generated by the data storage system based on the SQL statement and composed of several operator operations in a set logical order.

[0018] Optionally, the operator operation is a basic operation program generated by analyzing the SQL statement in the data storage system.

[0019] Optionally, the operator processing program is used to perform preprocessing operations on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement in the data storage system; wherein, the preprocessing operation is a processing operation that needs to be performed by the processing module after generating the execution plan.

[0020] Optionally, before executing the step of performing a pre-processing operation on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement, the following step is executed: performing a legality check on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement; the legality check is to determine whether the data structure of the operator operation of the SQL statement complies with predetermined SQL grammar rules.

[0021] Optionally, the syntax analysis result includes an operator operation processing result generated after the corresponding operator processing program has been called through the callback interface, as well as an operator operation without calling the callback interface.

[0022] Correspondingly, the present application also provides a syntax analysis method for SQL statements in a data storage system, including: receiving an input SQL statement; performing syntax analysis on the SQL statement to be executed in the data storage system; during the syntax analysis process, calling the corresponding operator handler by setting a callback interface for the generated operator operation; judging whether the syntax analysis is completed and whether the called operator handler is executed, and if all of the above judgments are yes, generating a syntax analysis result that has brought back the processing result of the operator handler; and processing the content of the output syntax analysis result.

[0023] Optionally, during the syntax analysis process, the corresponding operator handler is called by setting a callback interface for the generated operator operation, including: during the syntax analysis process to generate an execution plan, the corresponding operator handler is called by setting a callback interface for the generated operator operation, wherein the execution plan is an abstract syntax tree generated in the data storage system based on the SQL statement and composed of several operator operations in a set logical order.

[0024] Correspondingly, the present application also provides a syntax analysis device for SQL statements in a data storage system, characterized in that it includes: a receiving unit, a second syntax analysis unit, a second calling unit, a second judgment unit and a processing unit; the receiving unit is used to receive an input SQL statement; the second syntax analysis unit is used to perform syntax analysis on the SQL statement to be executed in the data storage system; the second calling unit is used to call the corresponding operator processing program by setting a callback interface for the generated operator operation during the process of syntax analysis of the SQL statement in the data storage system to generate an execution plan, wherein the execution plan is an abstract syntax tree composed of several operator operations in a set logical order generated according to the SQL statement in the data storage system; the second judgment unit is used to judge whether the syntax analysis is completed and whether the called operator processing program is executed. If all of the above judgments are yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated; the processing unit is used to process the content of the output syntax analysis result.

[0025] Optionally, the second calling unit is specifically used to: during the process of generating an execution plan through grammatical analysis, call the corresponding operator handler by setting a callback interface for the generated operator operation, wherein the execution plan is an abstract syntax tree composed of several operator operations in a set logical order generated in the data storage system according to the SQL statement.

[0026] Correspondingly, the present application also provides a syntax analysis system in a data processing system, comprising: the syntax analysis device in the data processing system described above, and the syntax analysis device for SQL statements in the data storage system described above.

[0027] Correspondingly, the present application also provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store a program of a grammatical analysis method in a data processing system. After the device is powered on and runs the program of the grammatical analysis method in the data processing system through the processor, the following steps are performed: performing grammatical analysis on the statements to be executed; during the grammatical analysis process, calling the corresponding operator processing program by setting a callback interface for the generated operator operation.

[0028] Correspondingly, the present application also provides a storage device that stores a program for a grammatical analysis method in a data processing system. The program is run by a processor to perform the following steps: performing grammatical analysis on the statements to be executed; during the grammatical analysis process, calling the corresponding operator processing program by setting a callback interface for the generated operator operation.

[0029] Compared with the prior art, this application has the following advantages:

[0030] By adopting the syntax analysis method in the data processing system described in this application, the operator processing program that processes the operator operations based on the abstract syntax tree in the data processing system can be pre-positioned to the abstract syntax tree generation stage, avoiding the generation of a large number of abstract syntax tree intermediate objects, thereby optimizing the syntax analysis efficiency of the data processing system and reducing the occupation of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flowchart of a method for grammatical analysis in a data processing system provided by an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of a grammatical analysis device in a data processing system provided by an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0034] Figure 4 A flowchart of a method for parsing SQL statements in a data storage system provided by an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of a grammatical analysis device for SQL statements in a data storage system provided by an embodiment of the present invention;

[0036] Figure 6A flowchart of a syntax analysis process in a data processing system in the prior art;

[0037] Figure 7 A flowchart of a grammatical analysis process in a data processing system provided by an embodiment of the present invention;

[0038] Figure 8 A complete flow chart of a method for grammatical analysis in a data processing system provided by an embodiment of the present invention;

[0039] Figure 9 A schematic diagram of setting a callback interface during a syntax analysis process based on an SQL statement provided by an embodiment of the present invention;

[0040] Figure 10 A flowchart of a prior art method for performing syntax analysis based on an INSERT statement in a data processing system;

[0041] Figure 11 The present invention provides a flowchart of performing syntax analysis based on an INSERT statement in a data processing system. DETAILED DESCRIPTION

[0042] The following description sets forth numerous specific details to facilitate a thorough understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific implementations disclosed below.

[0043] The following describes in detail the embodiment of the grammatical analysis method in the data processing system provided by the present invention. Figure 1 As shown, it is a flowchart of a syntax analysis method in a data processing system provided by an embodiment of the present invention.

[0044] The embodiment of the present invention can be implemented based on the SQL Parser (SQL parser) in the traditional data storage system. Figure 1 Describe the specific implementation steps.

[0045] Step S101: performing grammatical analysis on the statement to be executed.

[0046] In an embodiment of the present invention, the data processing system may refer to a data storage system. The statements to be executed may refer to SQL statements used to query and manage storage objects in the data storage system. For example, the SQL statements may include: a SELECT statement (select data operation statement) for retrieving data rows and columns from a data storage system table; an INSERT statement (insert data operation statement) for adding new data rows to a data storage system table; a DELETE statement (delete data operation statement) for deleting data rows from a data storage system table; an UPDATE statement (update data operation statement) for updating data in a data storage system table, etc.

[0047] In the embodiments of the present invention, syntax analysis is performed on statements to be executed, i.e., the SELECT, INSERT, DELETE, and UPDATE statements in the aforementioned SQL statements are analyzed to generate an AST (Abstract Syntax Tree) containing a large number of operator operations. The operator operations are basic operation procedures generated by analyzing the SQL statements in the data storage system.

[0048] like Figure 6 As shown, it is a flow chart of the syntax analysis process in a data processing system in the prior art. In the prior art, SQL Parser (SQL syntax analysis module) is an indispensable and important component in the database. At present, SQLParser is usually set at the front processing link of the entire processing flow of SQL statement syntax analysis. After the SQL statement is input into the data storage system, it is syntax analyzed to generate AST, and then the subsequent processing module in the data storage system processes it based on AST. This process is usually a waterfall model, that is, SQL Parser first generates AST, and then the subsequent data processing module traverses and processes based on the operator operations stored in each node of AST. It should be noted that after SQL Parser performs syntax analysis on the statement to be executed, the subsequent data processing module in the data storage system may refer to Processing Modules or Optimizer optimization program processing modules, etc.

[0049] The above-mentioned traditional processing method usually generates a large number of AST intermediate objects, resulting in the occupation of more data storage system resources. To this end, the present invention provides a syntax analysis method in a data processing system.

[0050] like Figure 7As shown, it is a flowchart of a syntax analysis process in a data processing system provided by an embodiment of the present invention. First, the SQL statement to be executed is subjected to syntax analysis, that is: through the SQL Parser calling callback interface, the operator processing operation in the operator processing module is called into the SQL Parser to implement the syntax analysis of the input SQL statement to generate an abstract syntax tree AST. In the syntax tree generation process described in the present invention, the lexical analysis stage of the operator processing operation stored in each node of the AST can be pre-placed to further compact the processing flow of SQL statement parsing and processing, reduce the generation of AST intermediate objects, and optimize the occupation of system resources.

[0051] Step S102: During the syntax analysis process, the corresponding operator processing program is called by setting a callback interface for the generated operator operation.

[0052] Step S101 performs syntax analysis on the statement to be executed, preparing for the invocation of the corresponding operator processing program during the syntax analysis process by setting callback interfaces for the generated operator operations. In step S102, different callback interfaces can be set for the generated operator operations according to the various syntax analysis stages in the SQL Parser, thereby invoking the corresponding operator processing program.

[0053] Specifically, in the embodiment of the present invention, during the execution plan formation stage of the SQL Parser in the data storage system performing grammatical analysis on the SQL statement to be executed, the corresponding operator processing program is called by setting a callback interface for the operator operation of the generated SQL statement, wherein the execution plan is an AST abstract syntax tree composed of several operator operations in a set logical order generated according to the SQL statement in the data storage system. The operator operation is a basic operation program generated by analyzing the SQL statement in the data storage system. The operator processing program is used to perform preprocessing operations on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement in the data storage system. The preprocessing operation is a processing operation that needs to be performed by the processing module after the execution plan is generated.

[0054] It should be noted that before performing the step of pre-processing the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement, the following steps are performed: performing a validity check on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement; the validity check is to determine whether the data structure of the operator operation of the SQL statement complies with the predetermined SQL grammar rules.

[0055] The SQL parser described in the present invention generates an AST abstract syntax tree for the corresponding SQL statement, and then traverses the operator operations stored in each node of the abstract syntax tree. When traversing to an operator operation, the callback API (callback interface) corresponding to each operator operation is called.

[0056] like Figure 9 As shown in FIG, it is a schematic diagram of setting a callback interface in a syntax analysis process based on SQL statements provided by an embodiment of the present invention.

[0057] The operator operations and setting callback interfaces described in the embodiments of the present invention are all implemented programmatically. For example, corresponding callback APIs are defined for the operator operations generated during the SELECT (delete statement) syntax analysis process, including: table callback (table callback interface) for the table operator operation, table callback (hint callback interface) for the hint operator operation, predicate callback (operation callback interface) for the predicate operator operation, join callback (join callback interface) for the join operator operation, group by callback (grouping callback interface) for the group by operator operation, order by callback (command callback interface) for the order by operator operation, having callback (preset condition callback interface) for the having operator operation, limit callback (limit callback interface) for the limit operator operation, and set op callback (set callback interface) for the set op operator operation.

[0058] After generating an abstract syntax tree for a SELECT statement, each node on the abstract syntax tree contains various operator operations. During the traversal of the abstract syntax tree, when a relevant operator operation is encountered, the callback API corresponding to the operator operation is called, and the operator processing program that the subsequent data processing module needs to process based on the AST is pre-positioned to the stage of generating operator operations in the syntax analysis. The callback interface set up specifically may include:

[0059] interface Select Handler{

[0060] void process Table(SQL Table table); / / Callback interface for table callback operator operation;

[0061] void process Hint(SQL Hint hint); / / Callback interface for hint operator operation;

[0062] void process Where(SQL Where where); / / Callback interface for predicate operator operation;

[0063] void process Join(SQL Join join); / / Callback interface for Join operator operation;

[0064] void process Group By(SQL Group By group By); / / Callback interface for Group By clause operator operation;

[0065] void process Order By(SQL Order By order By); / / Callback interface for Order By clause operator operation;

[0066] void process Having(SQL Having having); / / Callback interface for the Having clause operator operation;

[0067] void process Limit(SQL Limit limit); / / Callback interface for Limit clause operator operation;

[0068] void process Set Op(SQL Set Op set Op); / / For UNION, INTERSECT,

[0069] Callback interface for operator operations such as MINUS.

[0070] like Figure 8 , which is a complete flow chart of a method for grammatical analysis in a data processing system provided by an embodiment of the present invention. The embodiment of the present invention may further include the following steps: Step S103: Determining whether the grammatical analysis is complete and whether the called operator processing program has been executed. If both are true, generating a grammatical analysis result that has returned the processing result of the operator processing program.

[0071] In step S102, after calling the corresponding operator processing program by setting a callback interface for the generated operator operation during the parsing process, this step determines whether the parsing process is complete and whether the called operator processing program has been executed, thereby generating a parsing result that has returned the processing results of the operator processing program. The operator processing program is used to perform preprocessing operations on the data structure of the operator operation generated during the parsing of the SQL statement to be executed in the data storage system. The preprocessing operations are the processing operations required to be performed by the processing module after generating the execution plan.

[0072] The grammatical analysis results include operator operation processing results generated after the corresponding operator processing program has been called through the callback interface, as well as operator operations that did not call the callback interface. The grammatical analysis process may refer to the process of performing grammatical analysis based on the SQL statement to be executed in the data storage system to generate an AST abstract syntax tree.

[0073] In the syntax analysis process, the corresponding operator processing program is called by setting a callback interface for the generated operator operation, including:

[0074] The data structure of the operator operations generated during the syntax analysis of the executed SQL statement is subjected to a validity check; the validity check determines whether the data structure of the operator operations of the SQL statement conforms to the predetermined SQL syntax rules. During the execution plan formation phase of the syntax analysis of the SQL statement to be executed in the data storage system, the corresponding operator processing program is called by setting a callback interface for the operator operations of the generated SQL statement. The execution plan is an abstract syntax tree generated by the data storage system based on the SQL statement and composed of several operator operations in a predetermined logical order. The operator operations are the basic operation procedures generated by analyzing the SQL statement in the data storage system.

[0075] like Figure 10 and 11 As shown, they are respectively a flowchart of performing syntax analysis based on an INSERT statement in a data processing system in the prior art and a flowchart of performing syntax analysis based on an INSERT statement in a data processing system provided by an embodiment of the present invention.

[0076] The present invention provides a specific embodiment, using the syntax analysis process of an INSERT statement in SQL as an example to illustrate. The general processing steps include validating the validity of the operator operation and, after verification, updating the corresponding data structure. For example: INSERT INTO TVALUES(1,'aa'),(2,'bb'),,(3,'cc'); the processing process is as follows:

[0077] Initialize the data structure according to the INSERT statements required by the subsequent data processing module, for example:

[0078] INSERT column_1_values[…];

[0079] column_2_values[…];

[0080] …

[0081] column_N_values[…];

[0082] In this embodiment, the subsequent data processing module needs to perform subsequent data processing in the above manner, so the data structure of the above mode is defined;

[0083] The SQL parser parses the above INSERT statement and performs operator operation query when it encounters the target table t.

[0084] First, confirm that table t exists. If it does not exist, the process will report an error and end.

[0085] Obtain operator operation information for columns of table t. For example, in this embodiment, table t has two columns, COL_1 and COL_2. Obtain operator operation information for the columns, including column names and column data types.

[0086] The SQL parser continues to parse the VALUES clause, which has three rows of records.

[0087] The data structure is:

[0088] INSERT column_1_values[? ,? ,? ];

[0089] column_2_values[? ,? ,? ];

[0090] Combine the column position number and data type to verify the validity of the data recorded in VALUES. For example, if COL_1 is an integer type and COL_2 is a varchar type, then 1, 2, and 3 pass the verification and all meet the requirements of the integer type. 'aa', 'bb', and 'cc' pass the verification and all meet the requirements of the varchar type.

[0091] The final processed data structure is as follows:

[0092] INSERT column_1_values[1, 2, 3];

[0093] column_2_values['aa', 'bb', 'cc'].

[0094] By adopting the syntax analysis method in the data processing system described in this application, the operator processing program of the data processing system based on the abstract syntax tree can be pre-positioned to the abstract syntax tree generation stage, avoiding the generation of a large number of abstract syntax tree intermediate objects, thereby optimizing the syntax analysis efficiency of the data processing system and reducing the occupation of system resources.

[0095] Corresponding to the grammatical analysis method in the data processing system provided above, the present invention also provides a grammatical analysis device in the data processing system. Since the embodiment of the device is similar to the embodiment of the method described above, the description is relatively simple. For relevant details, please refer to the description of the embodiment of the method described above. The following description of the embodiment of the device is only illustrative. Please refer to Figure 2 As shown, it is a schematic diagram of a syntax analysis device in a data processing system provided by an embodiment of the present invention.

[0096] The grammatical analysis device in the data processing system according to the embodiment of the present invention includes the following parts:

[0097] The first syntax analysis unit 201 is used to perform syntax analysis on the statement to be executed.

[0098] In an embodiment of the present invention, the data processing system may refer to a data storage system. The statements to be executed may refer to SQL statements used to query and manage storage objects in the data storage system. For example, the SQL statements may include: a SELECT statement (select data operation statement) for retrieving data rows and columns from a data storage system table; an INSERT statement (insert data operation statement) for adding new data rows to a data storage system table; a DELETE statement (delete data operation statement) for deleting data rows from a data storage system table; an UPDATE statement (update data operation statement) for updating data in a data storage system table, etc.

[0099] In the embodiments of the present invention, syntax analysis is performed on statements to be executed, i.e., the SELECT, INSERT, DELETE, and UPDATE statements in the aforementioned SQL statements are analyzed to generate an AST (Abstract Syntax Tree) containing a large number of operator operations. The operator operations are basic operation procedures generated by analyzing the SQL statements in the data storage system.

[0100] like Figure 6 As shown, it is a flow chart of the syntax analysis process in a data processing system in the prior art. In the prior art, SQL Parser (SQL syntax analysis module) is an indispensable and important component in the database. At present, SQLParser is usually set at the front processing link of the entire processing flow of SQL statement syntax analysis. After the SQL statement is input into the data storage system, it is syntax analyzed to generate AST, and then the subsequent processing module in the data storage system processes it based on AST. This process is usually a waterfall model, that is, SQL Parser first generates AST, and then the subsequent data processing module traverses and processes based on the operator operations stored in each node of AST. It should be noted that after SQL Parser performs syntax analysis on the statement to be executed, the subsequent data processing module in the data storage system may refer to Processing Modules or Optimizer optimization program processing modules, etc.

[0101] The above-mentioned traditional processing method usually generates a large number of AST intermediate objects, resulting in the occupation of more data storage system resources. To this end, the present invention provides a syntax analysis method in a data processing system.

[0102] like Figure 7 As shown, it is a flowchart of a syntax analysis process in a data processing system provided by an embodiment of the present invention. First, the SQL statement to be executed is subjected to syntax analysis, that is: through the SQL Parser calling callback interface, the operator processing operation in the operator processing module is called into the SQL Parser to implement the syntax analysis of the input SQL statement to generate an abstract syntax tree AST. In the syntax tree generation process described in the present invention, the lexical analysis stage of the operator processing operation stored in each node of the AST can be pre-placed to further compact the processing flow of SQL statement parsing and processing, reduce the generation of AST intermediate objects, and optimize the occupation of system resources.

[0103] The first calling unit 202 is used to call the corresponding operator processing program by setting a callback interface for the generated operator operation during the syntax analysis process.

[0104] Specifically, in the embodiment of the present invention, during the execution plan formation stage of the SQL Parser in the data storage system performing grammatical analysis on the SQL statement to be executed, the corresponding operator processing program is called by setting a callback interface for the operator operation of the generated SQL statement, wherein the execution plan is an AST abstract syntax tree composed of several operator operations in a set logical order generated according to the SQL statement in the data storage system. The operator operation is a basic operation program generated by analyzing the SQL statement in the data storage system. The operator processing program is used to perform preprocessing operations on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement in the data storage system. The preprocessing operation is a processing operation that needs to be performed by the processing module after the execution plan is generated.

[0105] It should be noted that before performing the step of pre-processing the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement, the following steps are performed: performing a validity check on the data structure of the operator operation generated during the grammatical analysis of the executed SQL statement; the validity check is to determine whether the data structure of the operator operation of the SQL statement complies with the predetermined SQL grammar rules.

[0106] The SQL parser described in the present invention generates an AST abstract syntax tree for the corresponding SQL statement, and then traverses the operator operations stored in each node of the abstract syntax tree. When traversing to an operator operation, the callback API (callback interface) corresponding to each operator operation is called.

[0107] like Figure 9 As shown in FIG, it is a schematic diagram of setting a callback interface in a syntax analysis process based on SQL statements provided by an embodiment of the present invention.

[0108] The operator operations and setting callback interfaces described in the embodiments of the present invention are all implemented programmatically. For example, corresponding callback APIs are defined for the operator operations generated during the SELECT (delete statement) syntax analysis process, including: table callback (table callback interface) for the table operator operation, table callback (hint callback interface) for the hint operator operation, predicate callback (operation callback interface) for the predicate operator operation, join callback (join callback interface) for the join operator operation, group by callback (grouping callback interface) for the group by operator operation, order by callback (command callback interface) for the order by operator operation, having callback (preset condition callback interface) for the having operator operation, limit callback (limit callback interface) for the limit operator operation, and set op callback (set callback interface) for the set op operator operation.

[0109] After generating an abstract syntax tree for a SELECT statement, each node on the abstract syntax tree contains various operator operations. During the traversal of the abstract syntax tree, when a relevant operator operation is encountered, the callback API corresponding to the operator operation is called, and the operator processing program that the subsequent data processing module needs to process based on the AST is pre-positioned to the stage of generating operator operations in the syntax analysis. The callback interface set up specifically may include:

[0110] interface Select Handler{

[0111] void process Table(SQL Table table); / / Callback interface for table callback operator operation;

[0112] void process Hint(SQL Hint hint); / / Callback interface for hint operator operation;

[0113] void process Where(SQL Where where); / / Callback interface for predicate operator operation;

[0114] void process Join(SQL Join join); / / Callback interface for Join operator operation;

[0115] void process Group By(SQL Group By group By); / / Callback interface for Group By clause operator operation;

[0116] void process Order By(SQL Order By order By); / / Callback interface for Order By clause operator operation;

[0117] void process Having(SQL Having having); / / Callback interface for the Having clause operator operation;

[0118] void process Limit(SQL Limit limit); / / Callback interface for Limit clause operator operation;

[0119] void process Set Op(SQL Set Op set Op); / / For UNION, INTERSECT,

[0120] Callback interface for operator operations such as MINUS.

[0121] The embodiment of the present invention may further include: a first judgment unit 203. The first judgment unit 203 is used to judge whether the syntax analysis is completed and whether the called operator processing program is executed. If both of the above judgments are yes, a syntax analysis result that has returned the processing result of the operator processing program is generated.

[0122] The grammatical analysis results include operator operation processing results generated after the corresponding operator processing program has been called through the callback interface, as well as operator operations that did not call the callback interface. The grammatical analysis process may refer to the process of performing grammatical analysis based on the SQL statement to be executed in the data storage system to generate an AST abstract syntax tree.

[0123] In the syntax analysis process, the corresponding operator processing program is called by setting a callback interface for the generated operator operation, including:

[0124] The data structure of the operator operations generated during the syntax analysis of the executed SQL statement is subjected to a validity check; the validity check determines whether the data structure of the operator operations of the SQL statement conforms to the predetermined SQL syntax rules. During the execution plan formation phase of the syntax analysis of the SQL statement to be executed in the data storage system, the corresponding operator processing program is called by setting a callback interface for the operator operations of the generated SQL statement. The execution plan is an abstract syntax tree generated by the data storage system based on the SQL statement and composed of several operator operations in a predetermined logical order. The operator operations are the basic operation procedures generated by analyzing the SQL statement in the data storage system.

[0125] By using the syntax analysis device in the data processing system described in this application, the operator processing program of the data processing system based on the abstract syntax tree can be pre-positioned to the abstract syntax tree generation stage, avoiding the generation of a large number of abstract syntax tree intermediate objects, thereby optimizing the syntax analysis efficiency of the data processing system and reducing the occupation of system resources.

[0126] Corresponding to the syntax analysis method in the data processing system provided above, the present invention also provides an electronic device. Figure 3 As shown, it is a schematic diagram of an electronic device provided by an embodiment of the present invention.

[0127] The electronic device provided by the present invention specifically includes: a processor 302 and a memory 301. The memory 301 is used to store a program for a grammatical analysis method in a data processing system. After the device is powered on and the grammatical analysis method program in the data processing system is run by the processor 302, step 1 is executed: grammatical analysis is performed on the statement to be executed; and step 2 is executed: during the grammatical analysis process, the corresponding operator processing program is called by setting a callback interface for the generated operator operation.

[0128] Corresponding to the syntax analysis method in the data processing system provided above, the present invention also provides a storage device, which stores a program of the syntax analysis method in the data processing system. The program can be run by a processor to execute step one: perform syntax analysis on the statement to be executed; step two: during the syntax analysis process, call the corresponding operator processing program by setting a callback interface for the generated operator operation.

[0129] Corresponding to the syntax analysis method in the data processing system provided above, the present invention also provides a syntax analysis method for SQL statements in a data storage system. Since the embodiment of this method is similar to the above embodiment, the description is relatively simple. For relevant details, please refer to the description of the above embodiment. The following description of the embodiment is only for illustration. Please refer to Figure 4 As shown, it is a flowchart of a syntax analysis method for SQL statements in a data storage system provided by an embodiment of the present invention.

[0130] Step S401: receiving an input SQL statement.

[0131] Step S402: performing syntax analysis on the SQL statement to be executed in the data storage system.

[0132] Step S403: During the syntax analysis process, the corresponding operator processing program is called by setting a callback interface for the generated operator operation.

[0133] In an embodiment of the present invention, the corresponding operator processing program is called by setting a callback interface for the generated operator operation. The specific implementation method is: in the process of generating an execution plan through syntax analysis, the corresponding operator processing program is called by setting a callback interface for the generated operator operation, wherein the execution plan is an abstract syntax tree composed of several operator operations in a set logical order generated according to the SQL statement in the data storage system.

[0134] Step S404 , determining whether the syntax analysis is completed and whether the called operator processing program is executed. If both of the above determinations are yes, generating a syntax analysis result that has brought back the processing result of the operator processing program.

[0135] Step S405: Process the content of the output syntax analysis result.

[0136] By adopting the syntax analysis method for SQL statements in the data storage system described in this application, the operator processing program of the data storage system based on the abstract syntax tree can be pre-positioned to the abstract syntax tree generation stage, avoiding the generation of a large number of abstract syntax tree intermediate objects, thereby optimizing the syntax analysis efficiency of the data storage system and reducing the occupation of system resources.

[0137] Corresponding to the syntax analysis method for SQL statements in the data storage system provided above, the present invention also provides a syntax analysis device for SQL statements in the data storage system. Since the embodiment of the device is similar to the above method embodiment, the description is relatively simple. For relevant details, please refer to the description of the above method embodiment. The following description of the embodiment of the device is only illustrative. Please refer to Figure 5 As shown, it is a schematic diagram of a syntax analysis device for SQL statements in a data storage system provided by an embodiment of the present invention.

[0138] The grammatical analysis device for SQL statements in the data storage system according to the embodiment of the present invention includes the following parts:

[0139] The receiving unit 501 is configured to receive an input SQL statement.

[0140] The second syntax analysis unit 502 is used to perform syntax analysis on the SQL statements to be executed in the data storage system.

[0141] The second calling unit 503 is configured to call the corresponding operator processing program by setting a callback interface for the generated operator operation during the syntax analysis process.

[0142] In an embodiment of the present invention, the corresponding operator processing program is called by setting a callback interface for the generated operator operation. The specific implementation method is: in the process of generating an execution plan through syntax analysis, the corresponding operator processing program is called by setting a callback interface for the generated operator operation, wherein the execution plan is an abstract syntax tree composed of several operator operations in a set logical order generated according to the SQL statement in the data storage system.

[0143] The second judgment unit 504 is used to judge whether the syntax analysis is completed and whether the called operator processing program is executed. If both of the above judgments are yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated.

[0144] The processing unit 505 is configured to process the content of the output syntax analysis result.

[0145] By using the syntax analysis device for SQL statements in the data storage system described in this application, the operator processing program of the data storage system based on the abstract syntax tree can be pre-positioned to the abstract syntax tree generation stage, avoiding the generation of a large number of abstract syntax tree intermediate objects, thereby optimizing the syntax analysis efficiency of the data storage system and reducing the occupation of system resources.

[0146] Although the present invention is disclosed above in terms of preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined by the claims of the present invention.

[0147] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0148] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0149] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A method for grammatical analysis in a data processing system, wherein the data processing system includes a data storage system, characterized in that: include: Perform syntax analysis on the SQL statements to be executed; In the process of performing syntax analysis on an SQL statement to be executed in the data storage system to form an execution plan, a corresponding operator processing program is called by setting a callback interface for the operator operation of the generated SQL statement; wherein the execution plan is an abstract syntax tree composed of a plurality of operator operations in a set logical order, generated in the data storage system based on the SQL statement, and the operator operations are basic operation programs generated by analyzing the SQL statement in the data storage system; the operator processing program is used to perform pre-processing operations on the data structure of the operator operations generated in the process of performing syntax analysis on the SQL statement to be executed; After the syntax analysis is completed, the abstract syntax tree is traversed, and when an operator operation is traversed, the callback interface corresponding to the operator operation is called; It is determined whether the syntax analysis process is completed and whether the called operator processing program is executed. If both of the above determinations are yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated.

2. The method for grammatical analysis in a data processing system according to claim 1, wherein: in, The pre-processing operation is a processing operation that needs to be performed by the processing module after the execution plan is generated.

3. The method for grammatical analysis in a data processing system according to claim 1, wherein: Before performing the step of pre-processing the data structure of the operator operation generated during the syntax analysis of the SQL statement to be executed, perform the following steps: Perform a validity check on the data structure of the operator operation generated during the syntax analysis of the SQL statement to be executed; the validity check is to determine whether the data structure of the operator operation of the SQL statement conforms to the predetermined SQL syntax rules.

4. The method for grammatical analysis in a data processing system according to claim 1, wherein: The syntax analysis result includes the operator operation processing result generated after the corresponding operator processing program has been called through the callback interface, as well as the operator operation without calling the callback interface.

5. A device for grammatical analysis in a data processing system, the data processing system including a data storage system, characterized in that: include: a first syntax analysis unit and a first calling unit; The first syntax analysis unit is used to perform syntax analysis on the SQL statement to be executed; The first calling unit is configured to call a corresponding operator processing program by setting a callback interface for an operator operation of the generated SQL statement during a process of performing syntax analysis on the SQL statement to be executed in the data storage system to form an execution plan; wherein the execution plan is an abstract syntax tree composed of a plurality of operator operations in a set logical order, generated in the data storage system based on the SQL statement, and the operator operations are basic operation programs generated by analyzing the SQL statement in the data storage system; and the operator processing program is configured to perform pre-processing operations on a data structure of the operator operations generated during the syntax analysis of the SQL statement to be executed; After the syntax analysis is completed, the abstract syntax tree is traversed, and when the operator operation is traversed, the callback interface corresponding to each operator operation is called; The first judgment unit is used to judge whether the syntax analysis process is completed and whether the called operator processing program is executed. If the above judgments are both yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated.

6. A method for parsing SQL statements in a data storage system, characterized in that: include: Receive input SQL statements; Performing grammatical analysis on the SQL statements to be executed in the data storage system; During the process of generating an execution plan through syntax analysis, a corresponding operator processing program is called by setting a callback interface for the generated operator operation; wherein the execution plan is an abstract syntax tree composed of a plurality of operator operations in a set logical order generated in the data storage system based on the SQL statement, and the operator operation is a basic operation program generated by analyzing the SQL statement in the data storage system; the operator processing program is used to perform pre-processing operations on the data structure of the operator operation generated during the syntax analysis of the SQL statement to be executed; After the syntax analysis is completed, the abstract syntax tree is traversed, and when an operator operation is traversed, the callback interface corresponding to the operator operation is called; Determine whether the syntax analysis is completed and whether the called operator processing program is executed. If both of the above judgments are yes, generate a syntax analysis result that has brought back the processing result of the operator processing program; Process the content of the output syntax analysis result.

7. A grammatical analysis device for SQL statements in a data storage system, characterized in that: include: A receiving unit, a second syntax analyzing unit, a second calling unit, a second judging unit, and a processing unit; The receiving unit is used to receive an input SQL statement; The second syntax analysis unit is used to perform syntax analysis on the SQL statement to be executed in the data storage system; The second calling unit is configured to call a corresponding operator processing program by setting a callback interface for the generated operator operation during the process of generating an execution plan through syntax analysis; wherein the execution plan is an abstract syntax tree composed of a plurality of operator operations in a set logical order generated in the data storage system based on the SQL statement, and the operator operation is a basic operation program generated by analyzing the SQL statement in the data storage system; and the operator processing program is configured to perform preprocessing operations on the data structure of the operator operation generated during the syntax analysis of the SQL statement to be executed; After the syntax analysis is completed, the abstract syntax tree is traversed, and when an operator operation is traversed, the callback interface corresponding to the operator operation is called; The second judgment unit is used to judge whether the syntax analysis is completed and whether the called operator processing program is executed. If both of the above judgments are yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated; The processing unit is used to process the content of the output syntax analysis result.

8. A syntax analysis system in a data processing system, characterized in that: include: The device for syntax analysis in the data processing system described in claim 5, and the device for syntax analysis of SQL statements in the data storage system described in claim 7.

9. An electronic device, characterized in that: include: processor; as well as The memory is used to store a program of a grammatical analysis method in a data processing system. After the device is powered on and the program of the grammatical analysis method in the data processing system is run by the processor, the following steps are performed: Perform syntax analysis on the SQL statements to be executed; In the process of performing syntax analysis on an SQL statement to be executed in the data storage system to form an execution plan, a corresponding operator processing program is called by setting a callback interface for the operator operation of the generated SQL statement; wherein the execution plan is an abstract syntax tree composed of a plurality of operator operations in a set logical order, generated in the data storage system based on the SQL statement, and the operator operations are basic operation programs generated by analyzing the SQL statement in the data storage system; the operator processing program is used to perform pre-processing operations on the data structure of the operator operations generated in the process of performing syntax analysis on the SQL statement to be executed; After the syntax analysis is completed, the abstract syntax tree is traversed, and when an operator operation is traversed, the callback interface corresponding to the operator operation is called; It is determined whether the syntax analysis process is completed and whether the called operator processing program is executed. If both of the above determinations are yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated.

10. A storage device, characterized in that: A program for a syntax analysis method in a data processing system is stored, wherein the data processing system includes a data storage system. The program is executed by a processor to perform the following steps: Perform syntax analysis on the SQL statements to be executed; In the process of performing syntax analysis on an SQL statement to be executed in the data storage system to form an execution plan, a corresponding operator processing program is called by setting a callback interface for the operator operation of the generated SQL statement; wherein the execution plan is an abstract syntax tree composed of a plurality of operator operations in a set logical order, generated in the data storage system based on the SQL statement, and the operator operations are basic operation programs generated by analyzing the SQL statement in the data storage system; the operator processing program is used to perform pre-processing operations on the data structure of the operator operations generated in the process of performing syntax analysis on the SQL statement to be executed; After the syntax analysis is completed, the abstract syntax tree is traversed, and when an operator operation is traversed, the callback interface corresponding to the operator operation is called; It is determined whether the syntax analysis process is completed and whether the called operator processing program is executed. If both of the above determinations are yes, a syntax analysis result that has brought back the processing result of the operator processing program is generated.

Images