Local debugging method, system and device based on FlinkSQL and storage medium

Abstract

The invention relates to the field of computers, in particular to a FlinkSQL-based local debugging method, system and device and a storage medium, comprising an execution layer, an operation layer and a data layer; wherein the execution layer is provided with an IDE tool, and a system running program is started through a plug-in or an extended function of the execution layer; the running layer reads SQL statements in batches after a running program is started according to project parameters pre-configured by a user, carries out preprocessing by utilizing the SQL statements, receives the preprocessed SQL statements, and carries out lexical analysis and grammatical analysis in sequence; before the SQL statement is executed, replacing a placeholder in the SQL statement; replacing placeholders with actual environment variables, data source values or other parameters according to context information or configuration information; and executing the SQL operation according to the analyzed and preprocessed SQL statement and the replaced parameter value, and returning an execution result. Through the scheme, the FlinkSQL job can be developed, debugged and optimized in a local environment, and stable operation of the FlinkSQL job in a production environment is ensured.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more particularly, to a local debugging method, system, device, and storage medium based on FlinkSQL. Background Art

[0002] As a high-performance stream processing framework, Flink is widely used in the field of big data processing. FlinkSQL provided by it is a declarative query language that allows users to process stream data through SQL statements. However, in actual development, especially for developers who are not familiar with Java or Scala, it is difficult to quickly develop, debug, and locate problems. Therefore, how to efficiently perform local debugging of FlinkSQL has become an urgent problem to be solved.

[0003] In the prior art, although Flink also provides a Client for developers to use, there are some limitations. Developers need to input SQL statements one by one, which is not only inefficient but also unable to intuitively understand the detailed execution process, making it difficult to meet the needs of development and quick problem location.

[0004] The FlinkSQL Client is mainly used to submit the running of SQL tasks. In actual use, developers do need to manually input SQL statements and it is difficult to directly obtain the detailed execution process. This increases the difficulty of development and debugging, especially in complex data processing scenarios. Summary of the Invention

[0005] Embodiments of the present invention provide a local debugging method, system, device, and storage medium based on FlinkSQL to at least solve problems such as difficult batch reading, annotation interference, lack of targeted executors, and inflexible placeholder replacement during the processing of SQL statements in the IDE environment, thereby improving the efficiency and accuracy of SQL statement processing.

[0006] The purpose of the present invention is achieved by adopting the following technical solutions:

[0007] According to an embodiment of the present invention, a local debugging system based on FlinkSQL is provided. The system includes: an execution layer, a running layer, and a data layer; an IDE tool is provided in the execution layer, and the running layer is integrated in the IDE tool, and the system running program is started through a plugin or extended function of the IDE tool;

[0008] Among them, the running layer includes: an SQL reading module, which is used to batch read SQL statements according to project-related parameters pre-configured by the user after starting the running program; including the reading source of SQL statements, the text file path, or database connection information;

[0009] A cleaning module for reading SQL statements and performing preprocessing using the SQL statements;

[0010] An SQL parser for receiving the preprocessed SQL statements, performing lexical analysis and syntax analysis in sequence, and constructing an abstract syntax tree;

[0011] An SQL interceptor for replacing the placeholder in the SQL statement before executing the SQL statement; according to the context information or configuration information, replacing the placeholder with the actual environment variable, data source value or other parameters;

[0012] An SQL executor for executing SQL operations according to the parsed and preprocessed SQL statements and the replaced parameter values, and returning the execution results.

[0013] In an exemplary embodiment, the above cleaning module includes: a cleaner with the function of identifying and removing comments in SQL statements;

[0014] The cleaner is connected to the SQL parser for passing the preprocessed SQL statements to the SQL parser.

[0015] In an exemplary embodiment, the above SQL parser includes: an identification module for identifying SQL statement types, including CREATE, CTAS, SELECT, INSERT, SET, DROP types;

[0016] A creation module for dynamically creating corresponding executors according to the SQL statement types identified by the identification module; each type of executor is designed and optimized for its corresponding SQL statement.

[0017] In an exemplary embodiment, the above SQL executor includes: a legality verification module for verifying the received parsed and preprocessed SQL statements;

[0018] A scheduling module for obtaining the SQL statements that pass the verification by the legality verification module, generating corresponding execution plans according to the content and context information of the SQL statements, and scheduling corresponding tasks to execute SQL operations.

[0019] In an exemplary embodiment, the above data layer includes: a log recording module for real-time recording and printing out log information during the running of the program;

[0020] Wherein, the log information includes the execution process, execution results, and error information of the SQL statements.

[0021] According to an embodiment of the present invention, a local debugging method based on FlinkSQL is provided. The method includes: starting the system running program through a plug-in or extended function of an IDE tool;

[0022] After starting the running program, batch read SQL statements according to project-related parameters pre-configured by the user;

[0023] Perform preprocessing using the SQL statements;

[0024] Receive the preprocessed SQL statements, perform lexical analysis and syntax analysis in sequence, and construct an abstract syntax tree; before executing the SQL statements, perform replacement on the placeholder in the SQL statements; according to context information or configuration information, replace the placeholder with an actual environment variable, data source value, or other parameters;

[0025] Execute SQL operations according to the parsed and preprocessed SQL statements and the replaced parameter values, and return the execution results.

[0026] In an exemplary embodiment, the above-mentioned receiving the preprocessed SQL statements, performing lexical analysis and syntax analysis in sequence, and constructing an abstract syntax tree includes: identifying the SQL statement type, including CREATE, CTAS, SELECT, INSERT, SET, DROP types;

[0027] According to the identified SQL statement type, dynamically create corresponding executors, and each type of executor is designed and optimized for its corresponding SQL statement.

[0028] In an exemplary embodiment, the above-mentioned executing SQL operations and returning the execution results includes: verifying the received SQL statements that have been parsed and preprocessed;

[0029] Obtain the SQL statements that have passed the verification by the legality verification module, generate corresponding execution plans according to the content and context information of the SQL statements, and schedule corresponding tasks to execute the SQL operations.

[0030]

[0031] According to an embodiment of the present invention, a computer-readable storage medium is provided. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of the method described in any item of the second aspect are implemented.

[0032] According to an embodiment of the present invention, a computer device is provided. The computer device includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps of the method described in any item of the second aspect are implemented.​

[0033] The present invention provides a local debugging method, system, device and storage medium based on FlinkSQL, aiming to improve the development efficiency and accuracy of FlinkSQL jobs. It includes an execution layer, a running layer and a data layer. The execution layer is provided with an IDE tool, and the running layer is integrated in the IDE tool, and the system running program is started through a plug-in or extension function of the IDE tool. Among them, the running layer further includes: an SQL reading module, which is used to batch-read SQL statements according to the project-related parameters pre-configured by the user after the running program is started; including the reading source of the SQL statement, the text file path or the database connection information; a cleaning module, which is used to read the SQL statement and perform preprocessing using the SQL statement; an SQL parser, which is used to receive the preprocessed SQL statement, perform lexical analysis and syntax analysis in sequence, and construct an abstract syntax tree; an SQL interceptor, which is used to replace the placeholder in the SQL statement before executing the SQL statement; according to the context information or configuration information, replace the placeholder with the actual environment variable, data source value or other parameters;

[0034] An SQL executor, which is used to execute SQL operations according to the parsed and preprocessed SQL statement and the replaced parameter value, and return the execution result. Through the above solution, developers can develop, debug and optimize FlinkSQL jobs in a local environment to ensure their stable operation in a production environment.

[0035] In addition, the local debugging solution based on FlinkSQL proposed by the present invention also has the following advantages:

[0036] 1. Improve development efficiency: By supporting multi-SQL reading functions, it can read all SQL statements at one time, parse and execute them in sequence, reducing the time and effort for developers to input SQL statements one by one.

[0037] 2. Improve debugging accuracy: The SQL cleaner cleans the comments in the SQL, the parser parses the cleaned SQL statement and identifies the SQL type, the interceptor is responsible for replacing the placeholders in the SQL, replacing environment variables, etc., the executor executes the SQL statement, returns the result, and prints and outputs the log for easy analysis, enabling developers to more intuitively understand the execution process and result of the SQL statement, and timely discover and solve problems.

[0038] 3. Flexibly adapt to different usage scenarios: Support multiple reading source configurations, including the path of text files and the connection information of databases, etc., and can flexibly adapt to different usage scenarios., Ensure two-way data flow, enable each department of the enterprise to work collaboratively based on unified customer data, improve work efficiency, reduce information asymmetry, and enhance the overall operation efficiency of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0040] Figure 1 Schematic diagram of a local debugging system based on FlinkSQL provided by the present invention;

[0041] Figure 2 Flowchart of a local debugging method based on FlinkSQL provided by the present invention;

[0042] Figure 3 Block diagram of the hardware structure of a mobile terminal for the local debugging method based on FlinkSQL provided by the present invention;

[0043] Figure 4 Schematic diagram of the working principle of a local debugging method based on FlinkSQL provided by the present invention;

[0044] Figure 5 Schematic diagram of the structure of a computer device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the following further details the present application in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In this embodiment, a local debugging system based on FlinkSQL is provided. Figure 1 It is a schematic diagram of the structure of a local debugging system based on FlinkSQL according to an embodiment of the present invention. The system includes the following aspects: an execution layer, a running layer, and a data layer.

[0047] Among them, the execution layer mainly relies on the IDE to run, facilitating developers to output logs and debug problems.

[0048] The execution layer is provided with an IDE tool, and the running layer is integrated in the IDE tool, and the system running program is started through the plug-in or extension function of the IDE tool.

[0049] In one embodiment, the running layer includes: an SQL reading module, which is used to batch read SQL statements according to the project-related parameters pre-configured by the user after starting the running program; including the reading source of the SQL statement, the text file path, or the database connection information.

[0050] A cleaning module, which is used to read SQL statements and perform preprocessing using the SQL statements;

[0051] An SQL parser, which is used to receive the preprocessed SQL statements, perform lexical analysis and syntax analysis in sequence, and construct an abstract syntax tree;

[0052] An SQL interceptor, which is used to replace the placeholder in the SQL statement before executing the SQL statement; according to the context information or configuration information, replace the placeholder with the actual environment variable, data source value or other parameters;

[0053] An SQL executor, which is used to execute SQL operations according to the parsed and preprocessed SQL statements and the replaced parameter values, and return the execution results.

[0054] An SQL reading module. In the FlinkSql script or configuration file, the reading source of the SQL statement can be specified. This includes the path of the text file and the connection information of the database, etc., and supports multiple reading source configurations to flexibly adapt to different usage scenarios. The system supports the multi-SQL reading function, can read all SQL statements at one time, and parse and execute them in sequence. In addition to directly reading SQL statements from the reading source, it also allows dynamic generation of SQL statements through program logic.

[0055] In one embodiment, the cleaning module includes: a cleaner with the function of identifying and removing comments in SQL statements;

[0056] The cleaner is connected to the SQL parser and is used to pass the preprocessed SQL statements to the SQL parser.

[0057] In one embodiment, the SQL cleaning process is mainly responsible for cleaning the comments in the SQL text. Comments are mainly divided into two types: single-line comments and multi-line comments. Single-line comments start with "--", and all the content until the end of the line is regarded as a comment and removed. Multi-line comments start with " / " and end with " / ", and all the content between these two markers is regarded as a comment and removed. It should be noted that multi-line comments may be nested in other comments or strings, so when removing them, the boundary conditions need to be carefully processed to avoid accidentally deleting key information in the SQL statement, such as table names, column names, operators, etc. To ensure that these key information are not accidentally deleted, regular expressions or SQL parsers can be used to accurately match and retain this information. During the cleaning process, if an unrecognized comment format or other abnormal situations are encountered, these errors can be captured and corresponding processing can be carried out.

[0058] In one embodiment, the SQL parser includes: an identification module for identifying SQL statement types, including CREATE, CTAS, SELECT, INSERT, SET, DROP types;

[0059] A creation module for dynamically creating a corresponding executor according to the SQL statement type identified by the identification module; the executor of each type is designed and optimized for its corresponding SQL statement.

[0060] Further, in this embodiment, the SQL parser is mainly responsible for identifying and parsing various types of SQL statements, including but not limited to CREATE, CTAS (Create Table As Select), SELECT, INSERT, SET, DROP, etc. For each type of SQL statement, the parser will deeply understand its syntax structure and semantics for subsequent processing and analysis. The parser uses a syntax analyzer to perform lexical analysis and syntax analysis on the SQL statement and converts it into an Abstract Syntax Tree (AST) or other internal representation forms. During this process, the parser will strictly check the syntax correctness of the SQL statement and capture any syntax errors. In addition, the SQL parser can accurately identify the placeholders in the SQL statement. These placeholders are often used to represent values or parameters that need to be dynamically replaced. They can be variable names starting with specific characters or strings, or parameter placeholders in parameterized query statements. Once the placeholders are identified, the SQL interceptor (or related processing module) will replace these placeholders with actual environment variables or data source values according to the context information.

[0061] The SQL interceptor obtains actual environment variables or data source values from the configuration file, environment variables, database, or other data storage systems according to the current execution environment or configuration information. These values are used to replace the placeholders in the SQL statement. During the replacement process, the SQL interceptor will ensure the correctness and consistency of the data and strictly avoid introducing any potential security risks. To implement the replacement of placeholders, technical means such as a template engine or a string replacement algorithm can be used.

[0062] In the above embodiment, the SQL executor further includes: a legality verification module for verifying the received SQL statement after parsing and preprocessing;

[0063] A scheduling module for obtaining the SQL statement that has passed the verification by the legality verification module, generating a corresponding execution plan according to the content and context information of the SQL statement, and scheduling the corresponding task to execute the SQL operation.

[0064] Optionally, the SQL executor can define multiple executors according to the type of SQL statements, including Create executor, CTAS (Create Table As Select) executor, Select executor, Insert executor, Set executor, Drop executor, etc. Each executor is specifically designed and optimized for its corresponding SQL type to ensure efficient and accurate execution of relevant SQL operations. These executors will receive the SQL statements that have been parsed and preprocessed, and further verify them to ensure the correctness and legality of the statements. Then, according to the content and context information of the SQL statements, corresponding execution plans are generated, and corresponding tasks are scheduled to execute the SQL operations.

[0065] In one embodiment, the data layer includes: a log recording module for real-time recording and printing out log information during the running of the program;

[0066] Among them, the log information includes the execution process, execution result, and error information of the SQL statement.

[0067] During the running of the program, the program real-time records and prints out log information. The log information includes the execution process, execution result, error information, etc. of the SQL statement. By printing out the log, users can conveniently analyze the running status and performance, and timely discover and solve problems.

[0068] Based on the same inventive concept, the embodiment of the present application also provides a local debugging method based on FlinkSQL corresponding to a local debugging system based on FlinkSQL. The implementation solution provided by this method to solve problems is similar to the implementation solution recorded in the above system. Therefore, the specific limitations in one or more embodiments of the following local debugging methods based on FlinkSQL can refer to the limitations on the local debugging system based on FlinkSQL in the above text, and will not be repeated here.

[0069] In one embodiment, as Figure 2 shown, a local debugging method based on FlinkSQL provided by the embodiment of the present application specifically includes the following steps:

[0070] Step S11, start the system running program through the plug-in or extension function of the IDE tool;

[0071] Step S12, after starting the running program, batch read SQL statements according to the project-related parameters pre-configured by the user;

[0072] Step S13, perform preprocessing using the SQL statements;

[0073] Step S14: Receive the preprocessed SQL statement, perform lexical analysis and syntax analysis in sequence, and construct an abstract syntax tree; before executing the SQL statement, replace the placeholder in the SQL statement; according to the context information or configuration information, replace the placeholder with the actual environment variable, data source value or other parameters.

[0074] Step S15: Execute the SQL operation according to the parsed and preprocessed SQL statement and the replaced parameter value, and return the execution result.

[0075] In the above step S14, receiving the preprocessed SQL statement and performing lexical analysis and syntax analysis in sequence to construct an abstract syntax tree includes: identifying the SQL statement type, including types such as CREATE, CTAS, SELECT, INSERT, SET, DROP.

[0076] According to the identified SQL statement type, dynamically create the corresponding executor, and each type of executor is designed and optimized for its corresponding SQL statement.

[0077] In the above step S15, executing the SQL operation and returning the execution result includes: verifying the received SQL statement that has been parsed and preprocessed.

[0078] Obtain the SQL statement that has passed the verification by the legality verification module, generate the corresponding execution plan according to the content and context information of the SQL statement, and schedule the corresponding task to execute the SQL operation.

[0079] The method embodiments provided in the above embodiments of the present application can be executed on a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 3 is the hardware structure block diagram of the mobile terminal of the local debugging method based on FlinkSQL in the embodiments of the present invention. As Figure 3 shown, the mobile terminal may include one or more ( Figure 3 only one is shown in the figure) processors 101 (the processor 101 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 102 for storing data. Among them, the above mobile terminal may further include a transmission device 103 for communication functions and an input / output device 104. Those of ordinary skill in the art can understand that Figure 3 the structure shown in the figure is only schematic, and it does not limit the structure of the above mobile terminal. For example, the mobile terminal may further include more or fewer components than Figure 3 shown in the figure, or have a different configuration from Figure 3 shown in the figure.

[0080] The memory 102 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the abnormal prompt information of the node in the embodiment of the present invention. The processor 101 executes various functional applications and data processing by running the computer program stored in the memory 102, that is, implements the above-mentioned method. The memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 102 may further include a memory remotely disposed relative to the processor 101, and these remote memories can be connected to the mobile terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

[0081] The transmission device 103 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one instance, the transmission device 103 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one instance, the transmission device 103 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0082] To better illustrate the implementation manner of the present invention, the following gives a further description of the working principle of a specific local debugging method based on FlinkSQL, as Figure 4 shown.

[0083] 1. The IDE starts and runs the program, and reads SQL statements in batches

[0084] This SQL processing solution can be integrated into a mainstream integrated development environment (IDE), and the user starts and runs the program through the plug-in or extension function of the IDE. In the IDE, the user can configure relevant parameters of the project, including the reading source of SQL statements (such as text file path, database connection information, etc.). It supports reading SQL statements in batches, allowing the user to load multiple SQL statements into the memory at one time, which provides convenience for subsequent processing steps.

[0085] 2. Use the SQL cleaner to clean the comments in the SQL

[0086] After reading the SQL statements, first use the SQL cleaner to preprocess the SQL statements. The cleaner can identify and remove comments in the SQL statements, including single-line comments and multi-line comments, ensuring that the subsequent parsing and execution processes will not be interfered by comments, and improving the processing efficiency and accuracy.

[0087] 3. The parser parses the cleaned SQL statement and identifies the SQL type

[0088] The cleaned SQL statement is then passed to the parser. The parser performs lexical and syntactic analysis on the SQL statement to construct an Abstract Syntax Tree (AST) or other internal representation. Based on this, the parser further identifies the type of the SQL statement, such as CREATE, CTAS (Create Table As Select), SELECT, INSERT, SET, DROP, etc. Identifying the SQL type helps to adopt corresponding processing strategies according to different types of SQL statements.

[0089] 4. Create different SQL executors according to different SQL types

[0090] Based on the SQL type identified by the parser, corresponding SQL executors are dynamically created. Each type of executor is specifically designed and optimized for its corresponding SQL statement to ensure efficient and accurate execution of various SQL operations.

[0091] 5. The interceptor is responsible for replacing SQL placeholders, replacing environment variables, etc.

[0092] Before executing the SQL statement, the interceptor replaces the placeholders in the SQL statement. The interceptor replaces the placeholders with actual environment variables, data source values, or other parameters according to the context information or configuration information. This step ensures that the SQL statement can correctly reference the required data and resources when executed.

[0093] 6. The executor executes the SQL statement and returns the result

[0094] The executor executes the SQL operation based on the parsed and preprocessed SQL statement and the replaced parameter values.

[0095] 7. Print out logs for easy analysis

[0096] During the running of the program, the program records and prints out log information in real time. The log information includes the execution process, execution results, error information, etc. of the SQL statement. By printing out the logs, users can easily analyze the running status and performance and discover and solve problems in a timely manner.

[0097] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0098] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. Wherein, the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0099] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk or optical disk and other various media that can store computer programs.

[0100] An embodiment of the present invention further provides that in one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Figure 5 shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program, when executed by the processor, implements a local debugging method based on FlinkSQL.

[0101] Those skilled in the art can understand that Figure 5 the structure shown in

[0102] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary embodiments, and details thereof will not be repeated here.

[0103] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0104] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A local debugging system based on FlinkSQL, characterized in that: The system includes: an execution layer, a running layer and a data layer; the execution layer is provided with an IDE tool, the running layer is integrated into the IDE tool, and the system running program is started through the plug-in or extended function of the IDE tool; The operation layer includes: an SQL reading module, which is used to read SQL statements in batches after starting the operation program according to the project-related parameters pre-configured by the user; including the reading source of the SQL statements, text file path or database connection information; The cleaning module is used to read SQL statements and perform preprocessing using SQL statements; SQL parser, which receives the preprocessed SQL statement, performs lexical analysis and grammatical analysis in turn, and constructs an abstract syntax tree; SQL interceptor, used to replace the placeholders in SQL statements before executing them; based on context information or configuration information, the placeholders are replaced with actual environment variables, data source values, or other parameters; The SQL executor is used to execute SQL operations based on the parsed and preprocessed SQL statements and replaced parameter values, and return the execution results.

2. The system according to claim 1, characterized in that The cleaning module includes: a cleaner capable of identifying and removing comments in SQL statements; The cleaner is connected to the SQL parser and is used to transfer the pre-processed SQL statements to the SQL parser.

3. The system according to claim 2, characterized in that The SQL parser includes: an identification module for identifying SQL statement types, including CREATE, CTAS, SELECT, INSERT, SET, and DROP types; The creation module is used to dynamically create corresponding executors according to the SQL statement type identified by the recognition module; each type of executor is designed and optimized for its corresponding SQL statement.

4. The system according to claim 3, characterized in that The SQL executor includes: a legality verification module, which is used to verify the received SQL statement after parsing and preprocessing; The scheduling module is used to obtain the SQL statements that have passed the validation module, generate the corresponding execution plan according to the content and context information of the SQL statements, and schedule the corresponding tasks to execute the SQL operations.

5. The system according to claim 4, characterized in that The data layer includes: a log recording module, which is used to record and print out log information in real time during the operation of the program; The log information includes the execution process, execution results, and error information of the SQL statement.

6. A local debugging method based on FlinkSQL, characterized in that: The method comprises: starting a system running program through a plug-in or an extended function of an IDE tool; After starting the program, SQL statements are read in batches according to the project-related parameters pre-configured by the user; Use SQL statements for preprocessing; Receive the preprocessed SQL statement, perform lexical analysis and syntax analysis in sequence, and build an abstract syntax tree; before executing the SQL statement, replace the placeholders in the SQL statement; replace the placeholders with actual environment variables, data source values, or other parameters based on context information or configuration information; Execute SQL operations based on the parsed and preprocessed SQL statements and replaced parameter values, and return the execution results.

7. The method according to claim 6, characterized in that The receiving of the preprocessed SQL statement, performing lexical analysis and grammatical analysis in sequence, and constructing an abstract syntax tree includes: identifying the SQL statement type, including CREATE, CTAS, SELECT, INSERT, SET, and DROP types; According to the identified SQL statement type, the corresponding executor is dynamically created, and each type of executor is designed and optimized for its corresponding SQL statement.

8. The method according to claim 6, characterized in that The executing of the SQL operation and returning the execution result includes: verifying the received SQL statement after parsing and preprocessing; Obtain the SQL statements that have passed the validity verification module, generate the corresponding execution plan based on the content and context information of the SQL statements, and schedule the corresponding tasks to execute the SQL operations.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 6 to 8 when executed by a processor.

10. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 6 to 8 when executing the computer program.

Images