Structured statement execution method and device, storage medium and electronic equipment

Abstract

The invention provides a structured statement execution method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a structured statement, and checking the grammar correctness of the structured statement, the language type of the structured statement being one of the following: a data definition language, a data manipulation language, a data control language and a data query language; under the condition that the grammar of the structured statement is correct, analyzing the structured statement by adopting SQL Parser to obtain a hierarchical abstract syntax tree; determining the language type of the structured statement according to the hierarchical abstract syntax tree; and processing the structured statement according to the language type of the structured statement to obtain a target statement, and executing the target statement to perform corresponding operation on the ES cluster. According to the method, the SQL instruction is accurately converted into the domain specific language adapting to the ES internal working principle, the cost that an SQL user is familiar with ES native API and grammar is reduced, and deep integration of the SQL protocol and the ES is achieved.

Description

Technical Field

[0001] The present application relates to the field of big data processing and analysis, and specifically to a method for executing structured statements based on an ES cluster, an execution device for structured statements based on an ES cluster, a computer-readable storage medium, and an electronic device. Background Art

[0002] Currently, ElasticSearch, as a popular search and analysis engine, is widely used in the field of big data processing and analysis.

[0003] In traditional approaches, business personnel often have little or no development capabilities, requiring the assistance of developers throughout the development cycle. This consumes limited development resources and slows down overall development progress. Due to the ease of learning and versatility of structured languages, some business personnel have some basic familiarity with them. Furthermore, traditional batch processing methods take a long time to process and are unable to perform near-real-time queries such as keyword searches. Search methods based on a single system suffer from complex query language APIs and the inability to flexibly configure search conditions. These APIs and syntax differ from standard SQL, creating learning barriers for users accustomed to SQL when migrating to ElasticSearch.

[0004] Furthermore, while higher versions of ES support most DQL (data query language) operations, they lack support for DDL (data definition language) and DML (data manipulation language) in the SQL protocol. This requires users to learn and master ElasticSearch-specific APIs and DSL syntax when processing ElasticSearch data, increasing operational complexity and learning costs.

[0005] Therefore, existing technologies require operators to learn and master ElasticSearch's unique query language, which increases operational difficulty and labor costs, and cannot fully cover DDL and DML requirements. Especially in large-scale data environments, it lacks flexible and efficient data definition and manipulation functions. Summary of the Invention

[0006] The main purpose of this application is to provide an execution method for structured statements based on an ES cluster, an execution device for structured statements based on an ES cluster, a computer-readable storage medium and an electronic device, so as to at least solve the problem in the prior art that operators are required to learn and master the query language unique to ElasticSearch, which increases the difficulty of operation and labor costs.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for executing structured statements based on an ES cluster is provided, including: obtaining a structured statement and verifying the grammatical correctness of the structured statement, where the language type of the structured statement is one of the following: data definition language, data manipulation language, data control language and data query language; when the grammar of the structured statement is correct, using SQL Parser to parse the structured statement to obtain a hierarchical abstract syntax tree; determining the language type of the structured statement based on the hierarchical abstract syntax tree; processing the structured statement using a corresponding processing method based on the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and executing the target statement to perform corresponding operations on the ES cluster, where the target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

[0008] Optionally, SQL Parser is used to parse the structured statement to obtain a hierarchical abstract syntax tree, including: using Lexer based on the SQL Parser to read the structured statement, and identifying multiple continuous texts in the structured statement to obtain tokens corresponding to each continuous text, each of the continuous texts having corresponding features; using a deterministic finite automaton and a predefined lexical file to determine the value and type of each token; inputting all the tokens into a syntax analyzer, and using the syntax description of the structured statement as a rule to construct a hierarchical abstract syntax tree, wherein a leaf node of the hierarchical abstract syntax tree is part of a sub-statement in the structured statement.

[0009] Optionally, the language type of the structured statement is the data definition language, and the structured statement is processed in a corresponding manner according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster, including: when the structured statement is an SQL CREATE statement, parsing the index name and field definition in the SQL CREATE statement, and building a corresponding creation mapping JSON structure according to the index name and field definition in the SQL CREATE statement, and then using the PUT method to call the index creation interface of the ES cluster to create an index in the ES cluster according to the creation mapping JSON structure, and the creation mapping is used to define the document field structure and attributes in the ES cluster; when the structured statement is an SQL DROP statement, parsing the index name in the SQL DROP statement, and calling the index deletion interface of the ES cluster according to the index name in the SQLDROP statement to delete the corresponding index in the ES cluster; when the structured statement is an SQL TRUNCATE statement, parsing the index name in the SQL TRUNCATE statement, and building a corresponding creation mapping JSON structure according to the SQL The index name in the TRUNCATE statement constructs a corresponding document query, calls the document deletion interface of the ES cluster, and deletes all documents corresponding to the index name in the SQL TRUNCATE statement based on the document query.

[0010] Optionally, the language type of the structured statement is the data definition language, and the structured statement is processed in a corresponding manner according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster, including: when the structured statement is an SQL ALTER statement and is a field update, parsing the changed field content in the SQL ALTER statement, constructing an update mapping JSON structure corresponding to the changed field content, and calling the mapping update interface of the ES cluster through the PUT method to update the field in the ES cluster according to the update mapping JSON structure; when the structured statement is an SQL ALTER statement and is a field attribute update, using the Reindex API to create a new index, and then converting the old index data into new index data according to the content of the SQL ALTER statement, and then re-indexing the new index data into the new index.

[0011] Optionally, the language type of the structured statement is the data manipulation language, and the structured statement is processed in a corresponding manner according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster, including: when the structured statement is an SQL INSERT statement, extracting the document content and target index name in the SQL INSERT statement, converting the document content in the SQL INSERT statement into JSON format to obtain the target document content, and using the POST method to call the INDEX API of the ES cluster to insert the target index name and the target document content into the ES cluster; when the structured statement is an SQL UPDATE statement, extracting the update field, update value and target document ID in the SQL UPDATE statement, constructing a JSON structure including an update script based on the update field, the update value and the target document ID, and calling the UPDATE API of the ES cluster through the POST method to update the update script to the corresponding target document in the ES cluster, the update script including the update field and the update value; when the structured statement is an SQL DELETE statement, extracting the SQL The target document ID and index name in the DELETE statement are used to call the DELETE API of the ES cluster, and the corresponding target document is deleted from the ES cluster according to the target document ID and index name in the SQL DELETE statement.

[0012] Optionally, the language type of the structured statement is the data control language, and the structured statement is processed in a corresponding processing manner according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster, including: when the structured statement is an SQL GRANT statement, extracting the target user and corresponding user permission information in the SQL GRANT statement, and using the role management API to construct or update the user information and corresponding user permission information of the target user in the ES cluster according to the user permission information; when the structured statement is a CREATE USER statement, parsing the new user name and the corresponding user name, password and user permissions in the CREATE USER statement, using the create user endpoint of the user management API and creating a new user according to the user endpoint, the new user name in the CREATE USER statement and the corresponding user name, password and user permissions; when the structured statement is a DROP USER statement, parsing the DROP USER statement to obtain the deleted user name, and calling the delete user endpoint of the user management API to delete the user corresponding to the deleted user name.

[0013] Optionally, the language type of the structured statement is the data query language, and the structured statement is processed in a corresponding processing manner according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster, including: when the version of the ES cluster is greater than the preset version, executing the structured statement in a preset processing manner; when the version of the ES cluster is less than or equal to the preset version, parsing the structured statement to obtain extracted information, constructing a query DSL for the ES cluster based on the extracted information, and calling the search API of the ES cluster using the GET method, passing the constructed query DSL based on the search API to query the data in the ES cluster to obtain a JSON response returned by the ES cluster, parsing the JSON response, extracting and formatting the query results, and the query DSL includes: return fields and query conditions.

[0014] According to another aspect of the present application, an execution device for structured statements based on an ES cluster is provided, comprising: an acquisition unit for acquiring a structured statement and verifying the grammatical correctness of the structured statement, wherein the language type of the structured statement is one of the following: a data definition language, a data manipulation language, a data control language, and a data query language; a parsing unit for parsing the structured statement using SQL Parser to obtain a hierarchical abstract syntax tree when the syntax of the structured statement is correct; a determination unit for determining the language type of the structured statement based on the hierarchical abstract syntax tree; an execution unit for processing the structured statement using a corresponding processing method based on the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and executing the target statement to perform corresponding operations on the ES cluster, wherein the target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the methods for executing structured statements based on the ES cluster.

[0016] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include an execution method for executing any one of the ES cluster-based structured statements.

[0017] Applying the technical solution of the present application, the above-mentioned method for executing structured statements based on the ES cluster first obtains the structured statement and verifies the grammatical correctness of the structured statement. The language type of the structured statement is one of the following: data definition language, data manipulation language, data control language and data query language; then, when the grammar of the structured statement is correct, the SQL Parser is used to parse the structured statement to obtain a hierarchical abstract syntax tree; according to the hierarchical abstract syntax tree, the language type of the structured statement is determined; finally, the structured statement is processed according to the language type of the structured statement using a corresponding processing method to convert the format of the structured statement into a target format to obtain a target statement, and the target statement is executed to perform corresponding operations on the ES cluster. The target format is a format that the ES cluster can recognize, and the processing method corresponds one-to-one to the language type of the structured statement. This method focuses on bridging the compatibility gap between ElasticSearch and the SQL protocol, especially the support shortcomings for DDL, DCL and DML operations, and accurately converts SQL instructions into a domain-specific language (DSL) that adapts to the internal working principles of ElasticSearch. It reduces the cost for SQL users to become familiar with ElasticSearch's native API and syntax, simplifies the user operation process, and achieves deep integration of the SQL protocol and ElasticSearch. It solves the problem in existing technologies that require operators to learn and master ElasticSearch's unique query language, which increases operational difficulty and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0019] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for executing structured statements based on an ES cluster provided in an embodiment of the present application is shown;

[0020] Figure 2 A flow chart of a method for executing structured statements based on an ES cluster according to an embodiment of the present application is shown;

[0021] Figure 3 A flow chart of another method for executing structured statements based on an ES cluster according to an embodiment of the present application is shown;

[0022] Figure 4 The figure shows a structural block diagram of an execution device for structured statements based on an ES cluster provided according to an embodiment of the present application.

[0023] The above drawings include the following reference numerals:

[0024] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] For ease of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0029] Elasticsearch (ES): Elasticsearch is an open-source, full-text search engine based on Lucene, focused on providing real-time search and analysis capabilities in a distributed environment. Its core concept is to shard data across multiple computers to form a highly scalable cluster. Elasticsearch also supports automatic data balancing and fault tolerance to ensure high data availability and fast query capabilities.

[0030] Structured Query Language: SQL (Structured Query Language) is an abbreviation for Structured Query Language, a programming language used to manage relational database systems (RDBMS). SQL is a language designed for databases, allowing users to perform various operations. Based on their functionality, SQL is divided into four main categories: Data Definition Language (DDL), Data Manipulation Language (DML), Data Query Language (DQL), and Data Control Language (DCL).

[0031] Abstract Syntax Tree (AST): The Abstract Syntax Tree is used to convert structured text such as source code or query statements into a hierarchical internal representation. AST is converted from the input text by a Parser. A Parser usually consists of two parts: a Lexer (lexical analyzer) and a Parser (syntax analyzer). The Lexer is responsible for dividing the text into meaningful symbols (tokens), while the Parser combines these tokens into a tree structure, namely the Abstract Syntax Tree, according to grammatical rules. Use the predefined Visitors in the library to handle common SQL operations, or extend them to meet the needs of in-depth understanding and conversion of SQL statements in specific application scenarios. Through jsqlparser or Druid, developers can easily perform SQL parsing, modification, verification, and generation operations.

[0032] As described in the background, relational database systems in the prior art support transactional processing, have excellent data integrity and security, and are suitable for application scenarios that require highly structured data and complex queries. For example, ElasticSearch provides a rich set of API interfaces and query syntax to support data storage, search, and analysis. However, these APIs and syntax differ from the standard SQL language, making it difficult for users accustomed to using SQL to learn certain learning barriers when migrating to ElasticSearch. Specifically, the disadvantage of the prior art is that operators are required to learn and master the query language unique to ElasticSearch, which increases operational difficulty and labor costs.

[0033] In order to solve the problem in the prior art that operators need to learn and master the query language unique to ElasticSearch, which increases the difficulty of operation and labor costs, the embodiments of the present application provide a method for executing structured statements based on an ES cluster, an execution device for structured statements based on an ES cluster, a computer-readable storage medium and an electronic device.

[0034] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0035] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure diagram of a mobile terminal for executing a structured statement based on an ES cluster according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0036] Memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the method for executing structured statements based on an ES cluster in an embodiment of the present invention. Processor 102 executes the computer programs stored in memory 104 to execute various functional applications and data processing, thereby implementing the aforementioned method. Memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, which can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. Transmission device 106 is used to receive or transmit data via a network. Specific examples of such networks may include a wireless network provided by the mobile terminal's telecommunications provider. In one example, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] In this embodiment, a method for executing structured statements based on an ES cluster that runs on a mobile terminal, a computer terminal, or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] Figure 2 This is a flow chart of the execution method of the structured statement based on the ES cluster according to the embodiment of the present application. Figure 2 As shown, the method includes the following steps:

[0039] Step S201: obtaining a structured statement and verifying the grammatical correctness of the structured statement, wherein the language type of the structured statement is one of the following: data definition language, data manipulation language, data control language, and data query language;

[0040] Specifically, the structured statement is an SQL statement. If the SQL statement is verified to be correct, the subsequent steps are performed. If the SQL statement is incorrect, an error is output. Obtaining a structured statement and verifying its syntactic correctness involves core concepts in natural language processing and compilation theory, namely lexical analysis and syntactic analysis. This process is typically implemented in software development using a parser, which can be understood as a tool for converting structured text (in this case, an SQL statement) into a computer-processable internal representation (usually an Abstract Syntax Tree (AST)).

[0041] Before any operation is executed, the parser detects and reports syntax errors, preventing invalid queries or operations. Validated SQL statements can be converted to Elasticsearch or other database API calls, enabling cross-database interoperability. By providing a standard SQL interface, users can perform database operations without having to learn database-specific APIs, lowering the barrier to entry. Syntax validation and semantic analysis prevent illogical operations, reducing the risk of system failure.

[0042] Step S202: If the syntax of the structured statement is correct, parse the structured statement using SQL Parser to obtain a hierarchical abstract syntax tree.

[0043] Specifically, the SQL Parser is used to parse statements and generate a hierarchical Abstract Syntax Tree (AST), based on parsing techniques from compiler theory. Through lexical analysis and syntactic analysis, the parser converts the input SQL text into a tree-like internal representation, the Abstract Syntax Tree (AST). This tree structure reflects the grammatical structure and logical hierarchy of the SQL statement.

[0044] The AST is the output of the parsing process. It is a data structure used to represent the grammatical structure of an SQL statement. Each node in the AST represents an element in the grammatical structure, such as a keyword, table name, field name, operator, etc., and the hierarchical relationship between nodes reflects the structural relationship and execution order of the SQL statement. Through the AST, the system can more intuitively understand the intent of the SQL statement and perform subsequent logical processing. The AST not only contains the constituent elements of the SQL statement, but also represents the logical connection between elements through the parent-child relationship between nodes. For example, in the AST of the CREATE TABLE statement, CREATE and TABLE are top-level nodes, and the table field definitions are child nodes of the TABLE node.

[0045] AST provides a structured representation of SQL statements, making subsequent processing (such as conversion, execution, and optimization) more intuitive and efficient. Based on AST, the system can more easily understand the semantics of SQL statements and convert them into operations executable by specific database systems (such as Elasticsearch), achieving cross-database interoperability.

[0046] Step S203, determining the language type of the structured statement according to the hierarchical abstract syntax tree;

[0047] Specifically, the AST contains the structure and semantics of SQL statements. By analyzing the node types and structure of the AST, the system can identify the main functions and operation types of the statements. In addition to node types, the system also needs to understand the relationships between nodes and the attributes of each node to more accurately determine the statement type. For example, the AST of a CREATE TABLE statement will have nodes for defining the fields that create the table, while the AST of a SELECT statement will have nodes for data sources, filter conditions, and result fields. The system determines the statement type by analyzing these nodes and their attributes.

[0048] Step S204: Process the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

[0049] Specifically, the above steps convert SQL (Structured Query Language) statements into API calls or query syntax that Elasticsearch understands, based on their language type (DDL, DML, DCL, DQL). This conversion process requires a deep understanding of the SQL statement's intent and mapping it to the corresponding Elasticsearch functionality, while also taking into account the differences in syntax and functionality between Elasticsearch and SQL.

[0050] Allowing Elasticsearch to operate using SQL statements significantly improves its user compatibility and usability. Through conversion, Elasticsearch can execute SQL functions not originally supported, such as DDL and DCL operations, expanding its functional boundaries. The system can optimize converted statements to suit Elasticsearch's characteristics, such as using the bulk API to batch process DML statements, improving data operation efficiency.

[0051] The above-mentioned execution method of structured statements based on ES cluster in this application first obtains structured statements and verifies the grammatical correctness of structured statements. The language type of structured statements is one of the following: data definition language, data manipulation language, data control language and data query language; then, when the grammar of the structured statements is correct, SQLParser is used to parse the structured statements to obtain a hierarchical abstract syntax tree; according to the hierarchical abstract syntax tree, the language type of the structured statements is determined; finally, the structured statements are processed using a corresponding processing method according to the language type of the structured statements to convert the format of the structured statements into a target format to obtain a target statement, and the target statement is executed to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement. This method focuses on bridging the compatibility gap between ElasticSearch and the SQL protocol, especially the support shortcomings for DDL, DCL and DML operations, and accurately converting SQL instructions into a domain-specific language (DSL) that adapts to the internal working principles of ElasticSearch. It reduces the cost for SQL users to become familiar with ElasticSearch's native API and syntax, simplifies the user operation process, and achieves deep integration of the SQL protocol and ElasticSearch. It solves the problem in existing technologies that require operators to learn and master ElasticSearch's unique query language, which increases operational difficulty and labor costs.

[0052] In some embodiments, using SQL Parser to parse the structured statement to obtain a hierarchical abstract syntax tree includes the following steps:

[0053] Step S2021: Read the structured statement using a Lexer based on the SQL Parser, and identify multiple continuous texts in the structured statement to obtain tokens corresponding to each of the continuous texts, wherein each of the continuous texts has corresponding features;

[0054] Step S2022: using a deterministic finite automaton and a predefined lexical file to determine the value and type of each token;

[0055] In step S2023, all the tokens are input into a syntax analyzer, and a hierarchical abstract syntax tree is constructed using the syntax description of the structured statement as a rule, wherein a leaf node of the hierarchical abstract syntax tree is a part of a sub-statement in the structured statement.

[0056] Specifically, through lexical and syntactic analysis, the system can detect the grammatical integrity of SQL statements. The constructed AST provides a clear semantic representation of SQL statements, which facilitates subsequent statement conversion and execution. The automated processing of lexical and syntactic analysis improves the parsing efficiency of SQL statements and reduces the need for manual intervention. In the connection between SQL and ElasticSearch, ElasticSearch-specific data types (such as geo_point, nest) and reserved keywords are specially processed to ensure that they are correctly identified and converted into corresponding ElasticSearch operation elements during the SQL lexical analysis stage, expanding the boundaries of data types supported by the SQL protocol for ElasticSearch and improving the practicality and adaptability of the system.

[0057] The SQL statement parsing part is divided into two parts: lexical parsing and grammatical parsing. SQL Parser can perform parsing work in both lexical parsing and grammatical parsing. During lexical analysis, Lexer continuously reads continuous SQL text, identifies a section of continuous text with certain characteristics as a token, and identifies the category of the token. In particular, it is necessary to repeatedly compare strings when identifying identifiers (variables, table names, column names, etc.) and reserved words (TABLE, FROM, SELECT, etc.). The automatic generation tool uses DFA (Deterministic Finite Automaton) and pre-defined lexical files at this stage to determine the value and type of each token. Manually written parsers do not require the maintenance of an additional state machine, and use branch prediction to reduce the amount of calculation and the depth of the call stack.

[0058] SQL statements do not support ElasticSearch data types such as geo_point and nest, or reserved keywords such as kibana_admin. Therefore, when these reserved keywords exist, they must be identified during the lexical analysis phase and marked as special token types rather than being treated as unrecognized words.

[0059] The parser uses the tokens from the lexical analysis as input, uses the SQL grammar description as rules, expands non-leaf nodes from top to bottom, builds a syntax tree, and generates a correct AST.

[0060] In some embodiments, the language type of the structured statement is the data definition language (DDL). Processing the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and executing the target statement to perform a corresponding operation on the ES cluster includes the following steps:

[0061] Step S301: When the structured statement is an SQL CREATE statement, parse the index name and field definition in the SQL CREATE statement, and construct a corresponding creation mapping JSON structure according to the index name and field definition in the SQL CREATE statement. Then, use the PUT method to call the index creation interface of the ES cluster to create an index in the ES cluster according to the creation mapping JSON structure. The creation mapping is used to define the document field structure and attributes in the ES cluster.

[0062] Specifically, it parses the index name and field definitions in the SQL CREATE statement, which is typically used to create an index or table. It then constructs the corresponding ElasticSearch mapping JSON structure. Then, it calls the ElasticSearch index creation API using the PUT method, passing in the index name and mapping structure. The PUT method processes the entire index, and the index mapping field types correspond to the data types supported by ElasticSearch, including support for special ElasticSearch types.

[0063] Step S302: If the structured statement is an SQL DROP statement, parse the index name in the SQL DROP statement, and call the index deletion interface of the ES cluster according to the index name in the SQL DROP statement to delete the corresponding index in the ES cluster;

[0064] Specifically, parse the index name in the SQL DROP statement and use the DELETE method to directly call the ElasticSearch index deletion interface, that is, directly delete the index.

[0065] Step S303, when the structured statement is an SQL TRUNCATE statement, parse the index name in the SQL TRUNCATE statement, build a corresponding document query based on the index name in the SQL TRUNCATE statement, call the document deletion interface of the ES cluster and delete all documents corresponding to the index name in the SQL TRUNCATE statement based on the document query.

[0066] Specifically, it parses the index name in the SQL TRUNCATE statement, builds a query that matches all documents, and then uses the DELETE method to call the ElasticSearch document deletion interface to delete all documents in the index, that is, to delete the data in the table but retain the table structure.

[0067] The above steps convert SQL DDL operations into Elasticsearch index management operations via the RESTful API, standardizing operations across different database systems. Users can continue to use the familiar SQL language for operations such as database creation, deletion, and purging, without having to learn Elasticsearch's specific API and DSL syntax. This translation layer ensures better compatibility with the needs of SQL users while improving the operational convenience of Elasticsearch clusters.

[0068] DDL is a language used to define database structures, such as creating, dropping, and clearing tables. JSON maps to a configuration file in Elasticsearch that defines the structure and properties of fields in an index (similar to a table). SQL CREATE, DROP, and TRUNCATE statements are all based on the SQL-DDL data definition language.

[0069] In some embodiments, the language type of the structured statement is the data definition language (DDL). Processing the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and executing the target statement to perform a corresponding operation on the ES cluster includes the following steps:

[0070] Step S401: If the structured statement is an SQL ALTER statement and is a field update, parse the changed field content in the SQL ALTER statement, construct an update mapping JSON structure corresponding to the changed field content, and call the mapping update interface of the ES cluster through the PUT method to update the field in the ES cluster according to the update mapping JSON structure;

[0071] Step S402, when the structured statement is an SQL ALTER statement and is a field attribute update, a new index is created using the Reindex API, and then the old index data is converted into new index data according to the content of the SQL ALTER statement, and the new index data is re-indexed into the new index.

[0072] Specifically, by converting SQL ALTER statements into Elasticsearch mapping updates or data reindexing operations, we enhance the flexibility of field and data attribute modifications. When handling complex field attribute updates, we ensure data consistency and integrity by switching between old and new indexes. This allows users to continue using SQL statements for DDL operations, lowering the learning curve and improving the system's user-friendliness.

[0073] The SQL ALTER statement is the SQL-DDL data definition language, and the RESTful API is a series of HTTP-based APIs provided by Elasticsearch for managing indexes and documents. For simple field attribute updates, the SQL ALTER statement is parsed to construct a JSON structure for the updated mapping, and the mapping update API is called using the PUT method. For complex structural changes, the Reindex API is used to modify, add, and change field attributes. A new index is created, and then the old index data is converted and re-indexed into the new index.

[0074] In some embodiments, the language type of the structured statement is the data manipulation language (DML). Processing the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and executing the target statement to perform a corresponding operation on the ES cluster includes the following steps:

[0075] Step S501: When the structured statement is an SQL INSERT statement, extract the document content and target index name in the SQL INSERT statement, convert the document content in the SQL INSERT statement into JSON format to obtain the target document content, and use the POST method to call the INDEX API of the ES cluster to insert the target index name and the target document content into the ES cluster.

[0076] Specifically, it parses the SQL INSERT statement, extracts the document content and the target index name, converts the document content into JSON format, and then uses the POST method to call the ElasticSearch INDEX API, passing in the index name and the document content in JSON format to perform the insert operation.

[0077] Step S502: If the structured statement is an SQL UPDATE statement, extract the update fields, update values, and target document ID in the SQL UPDATE statement, construct a JSON structure including an update script based on the update fields, update values, and target document ID, and call the UPDATE API of the ES cluster via the POST method to update the update script to the corresponding target document in the ES cluster, where the update script includes the update fields and update values.

[0078] Specifically, it parses the SQL UPDATE statement, extracts the fields to be updated, the new values, and the target document ID. It then constructs a JSON structure containing the update script and calls the ElasticSearch UPDATE API via the POST method, passing in the index name, document ID, and update script to execute the update.

[0079] Step S503, when the structured statement is an SQL DELETE statement, extract the target document ID and index name in the SQL DELETE statement, call the DELETE API of the ES cluster, and delete the corresponding target document from the ES cluster according to the target document ID and index name in the SQL DELETE statement.

[0080] Specifically, the SQL DELETE statement is parsed to obtain the target document ID and index name. The DELETE method is used to call the ElasticSearch DELETE API, passing in the index name and document ID to delete the specified document.

[0081] Specifically, users can use familiar SQL DML statements to operate Elasticsearch clusters, maintaining operational consistency and user-friendliness. REST API calls enable efficient data insertion, updating, and deletion, leveraging the distributed nature of Elasticsearch. The system can more accurately identify and handle errors related to DML operations and provide corresponding feedback.

[0082] Among them, SQL INSERT statements, SQL UPDATE statements, and SQL DELETE statements are all SQL-DML data manipulation languages, and SQL-DML data manipulation languages ​​generally edit rows or document contents.

[0083] In some embodiments, the language type of the structured statement is the Data Control Language (DCL). Processing the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and executing the target statement to perform a corresponding operation on the ES cluster includes the following steps:

[0084] Step S601: If the structured statement is a SQL GRANT statement, extract the target user and corresponding user permission information in the SQL GRANT statement, and use the role management API to construct or update the user information and corresponding user permission information of the target user in the ES cluster based on the user permission information.

[0085] This involves parsing the SQL GRANT statement to extract the permissions and target user / role. Based on the permissions information, a role definition in ElasticSearch is constructed or updated, and the role management API is used to create or update the role. For specific users, the user management API is also called to associate the user with the role.

[0086] Step S602: If the structured statement is a CREATE USER statement, the newly created user name, the corresponding user name, password, and user permissions in the CREATE USER statement are parsed, and a new user is created based on the user endpoint, the newly created user name, the corresponding user name, password, and user permissions in the CREATE USER statement using the create user endpoint of the user management API.

[0087] For CREATE USER, the SQL statement is parsed to extract the user name, password, and permissions, and the user is created using the Create User endpoint of the User Management API.

[0088] Step S603: If the structured statement is a DROP USER statement, the DROP USER statement is parsed to obtain a deleted user name, and a delete user endpoint of the user management API is called to delete the user corresponding to the deleted user name.

[0089] For DROP USER, the SQL statement is parsed to extract the user name, and the delete user endpoint of the user management API is called to delete the user.

[0090] Specifically, by converting SQL statements like GRANT, CREATE USER, and DROP USER into Elasticsearch's RESTful API calls, a standardized interface is provided for user permission management, enabling SQL users to manage permissions and users without having to learn Elasticsearch-specific APIs. Whether in a relational database or Elasticsearch, users can use consistent DCL statements to control data access, significantly improving cross-system operational consistency and user experience. This conversion layer enables the system to process SQL DCL statements, enabling user and permission management for Elasticsearch, and enhancing compatibility and acceptance of the Elasticsearch system among SQL users.

[0091] The system automatically parses SQL DCL statements, extracts relevant parameters (such as username, password, and permissions), and calls the corresponding Elasticsearch API for processing. This automated conversion and execution reduces the need for manual configuration and intervention, improving the efficiency of user and permissions management. This automated process reduces the likelihood of manual errors and ensures the accuracy and consistency of permissions settings.

[0092] In some embodiments, the language type of the structured statement is the Data Query Language (DQL). Processing the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and executing the target statement to perform a corresponding operation on the ES cluster includes the following steps:

[0093] Step S701: If the version of the ES cluster is greater than a preset version, execute the structured statement in a preset processing mode.

[0094] Among them, the preset processing method is the existing processing method, and the preset version is ES7.X. Elasticsearch continuously optimizes and expands its functions between different versions. For example, starting with Elasticsearch 7.x, a more stable SQL query interface has been introduced in the existing technology, allowing data to be queried directly using SQL statements through the REST API. This means that higher versions of ES have more mature and direct support for the SQL protocol. The SQL query interface of higher versions of ES provides a more efficient and stable query experience, including but not limited to better support for aggregation, window functions, etc., as well as more comprehensive compatibility with SQL syntax. This makes it possible to use SQL statements directly without the need for additional conversion steps.

[0095] Directly using the SQL query interface reduces the overhead associated with converting SQL statements to a DSL and improves query speed. Furthermore, higher versions of ES theoretically parse and execute SQL statements more accurately, reducing potential conversion errors. Directly using the SQL query interface provided by existing technologies reduces the development and maintenance of custom conversion logic, lowering system complexity and maintenance costs.

[0096] Step S702: When the version of the ES cluster is less than or equal to the preset version, parse the structured statement to obtain extracted information, construct the query DSL of the ES cluster based on the extracted information, and use the GET method to call the search API of the ES cluster. Based on the search API, pass the constructed query DSL to query the data in the ES cluster to obtain the JSON response returned by the ES cluster, parse the JSON response, extract and format the query results, and the query DSL includes: return fields and query conditions.

[0097] Specifically, for ES clusters with versions lower than or equal to the preset version, the system parses SQL structured statements, extracts key information (such as return fields and query conditions), converts this information into a query DSL (Domain Specific Language) recognizable by ES, and then executes the query by calling the search API. Although this method is not optimal for higher ES versions, it ensures that DQL statements can be effectively executed in all ES versions, demonstrating the system's wide applicability and version compatibility.

[0098] For versions later than ES7.X, queries are performed using the _sql API provided by existing technologies. For versions earlier than ES7.X, simple query requests are supported. Based on the extracted information, the ElasticSearch query DSL is constructed, including specifying return fields and setting query conditions. The GET method is used to call the ElasticSearch _search API, passing in the constructed query DSL. The JSON response returned by ElasticSearch is parsed to extract and format the query results.

[0099] Directly executing SQL queries in higher-level ES versions avoids the need for SQL-to-DSL conversion, significantly reducing the potential syntax and logic conversion overhead during the query process and improving query speed and accuracy. In earlier ES versions, by meticulously parsing SQL statements, the system accurately constructs the DSL query structure, including specified return fields and complex query conditions. This precision helps ensure the accuracy of query results, while the flexibility and high-performance features of the DSL also ensure query efficiency.

[0100] Regardless of the ES cluster version, users can use unified SQL syntax to query data, which greatly improves the user-friendliness of the system and reduces the learning curve and operational complexity.

[0101] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for executing structured statements based on the ES cluster of the present application will be described in detail below with reference to specific embodiments.

[0102] This embodiment relates to a specific method for executing structured statements based on ES clusters, such as Figure 3 As shown, the following steps are included:

[0103] Step S1: using the RestFul API input structure to receive structured SQL statements, for example: CREATE TABLE IFNOT EXISTS STUDENT('NAME' TEXT COMMENT name), TRUNCATE TABLE TABLE_NAME;

[0104] Step S2: Verify the syntax correctness of the input structured statement (SQL), parse the SQL statement through SQLParser to obtain a hierarchical abstract syntax tree, including but not limited to the following processing: word analyzer, syntax analyzer, tree analyzer and parameter extraction;

[0105] Step S3: Based on the abstract syntax tree results, the structured statements are converted into the syntax of the target ElasticSearch search and analysis engine. Specifically, based on the abstract syntax tree results, the types of structured statements are determined, such as data definition language (CREATE, ALTER, DROP, and TRUNCATE), data manipulation language (INSERT, UPDATE, and DELETE syntax), data query language (SELECT), and data control language (GRANT, CREATE / DROP USER).

[0106] For data definition languages, the ES interface request body is spliced ​​based on the operation type and parameters obtained through SQL parsing. For data manipulation languages, the index name is extracted from SQL and converted into a POST request interface. For data query languages, ES7.X versions support the conversion of SQL query statements to DSL, and the converted query is used to query distributed clusters. For data control languages, the created user or the role authorized to the user is identified.

[0107] Step S4: Execute the converted statement to perform corresponding operations on the ES cluster, such as adding fields, deleting fields, deleting table data, and deleting tables.

[0108] The above examples use the SQLParser component to parse input SQL statements and convert them into operations for the ES cluster, thereby supporting ElasticSearch's DDL and DML operations. This improves the efficiency and accuracy of storing and analyzing ElasticSearch data and reduces labor costs.

[0109] The seamless SQL-ElasticSearch integration focuses on bridging the compatibility gap between ElasticSearch and the SQL protocol, particularly the support for DDL, DCL, and DML operations. It precisely converts SQL commands into a domain-specific language (DSL) that adapts to the internal workings of ElasticSearch. This reduces the cost for SQL users to become familiar with ElasticSearch's native API and syntax, simplifies user operations, and achieves deep integration between the SQL protocol and ElasticSearch.

[0110] The above-mentioned embodiments will greatly improve the operational efficiency and accuracy of the ElasticSearch database and reduce the labor costs of storing and analyzing data. At the same time, they also have good versatility and ease of use and can be widely applied to various scenarios that require processing large amounts of data. The ElasticSearch database operation conversion system based on the SQL protocol solves the problem that ElasticSearch does not support DDL statements, DCL statements, and DML statements in the SQL protocol, and provides a method for converting DDL and other statements into operations that can be recognized and executed by ElasticSearch. It has broad application prospects and market value.

[0111] The embodiment of the present application also provides an execution device for structured statements based on ES clusters. It should be noted that the execution device for structured statements based on ES clusters in the embodiment of the present application can be used to execute the execution method for structured statements based on ES clusters provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation methods, and those that have been explained will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0112] The following introduces the execution device of structured statements based on ES cluster provided in an embodiment of the present application.

[0113] Figure 4 Schematic diagram of an execution device for structured statements based on an ES cluster according to an embodiment of the present application. Figure 4 As shown, the device includes an acquisition unit 10, a parsing unit 20, a determination unit 30 and an execution unit 40. The acquisition unit 10 is used to acquire a structured statement and verify the grammatical correctness of the structured statement. The language type of the structured statement is one of the following: data definition language, data manipulation language, data control language and data query language; the parsing unit 20 is used to parse the structured statement using SQL Parser to obtain a hierarchical abstract syntax tree when the syntax of the structured statement is correct; the determination unit 30 is used to determine the language type of the structured statement according to the hierarchical abstract syntax tree; the execution unit 40 is used to process the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

[0114] The above-mentioned execution device of structured statements based on ES cluster of the present application includes an acquisition unit, a parsing unit, a determination unit and an execution unit. The acquisition unit is used to acquire structured statements and verify the grammatical correctness of structured statements. The language type of structured statements is one of the following: data definition language, data manipulation language, data control language and data query language; the parsing unit is used to parse the structured statements using SQL Parser when the grammar of the structured statements is correct to obtain a hierarchical abstract syntax tree; the determination unit is used to determine the language type of the structured statements according to the hierarchical abstract syntax tree; the execution unit is used to process the structured statements using a corresponding processing method according to the language type of the structured statements to convert the format of the structured statements into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds to the language type of the structured statement. The device focuses on bridging the compatibility gap between ElasticSearch and the SQL protocol, especially the support shortcomings for DDL, DCL and DML operations, and accurately converts SQL instructions into a domain-specific language (DSL) that adapts to the internal working principles of ElasticSearch. It reduces the cost for SQL users to become familiar with ElasticSearch's native API and syntax, simplifies the user operation process, and achieves deep integration of the SQL protocol and ElasticSearch. It solves the problem in existing technologies that require operators to learn and master ElasticSearch's unique query language, which increases operational difficulty and labor costs.

[0115] In some embodiments, the parsing unit includes an identification module, a first determination module and a construction module, the identification module is used to read the structured statement based on the SQL Parser using Lexer, and identify multiple continuous texts in the structured statement to obtain tokens corresponding to each continuous text, and each continuous text has corresponding features; the first determination module is used to use a deterministic finite automaton and a predefined lexical file to determine the value and type of each token; the construction module is used to input all the tokens into the syntax analyzer, and use the syntax description of the structured statement as a rule to construct a hierarchical abstract syntax tree, where a leaf node of the hierarchical abstract syntax tree is part of a substatement in the structured statement. Through lexical and grammatical analysis, the system can detect the grammatical integrity of the SQL statement. The constructed AST provides a clear semantic representation of the SQL statement, which facilitates subsequent statement conversion and execution.

[0116] In some embodiments, the language type of the structured statement is the data definition language, and the execution unit includes a first parsing module, a second parsing module, and a third parsing module. The first parsing module is used to parse the index name and field definition in the SQL CREATE statement when the structured statement is a SQL CREATE statement, and build a corresponding creation mapping JSON structure according to the index name and field definition in the SQL CREATE statement, and then use the PUT method to call the index creation interface of the ES cluster to create an index in the ES cluster according to the creation mapping JSON structure, and the creation mapping is used to define the document field structure and attributes in the ES cluster; the second parsing module is used to parse the index name in the SQL DROP statement when the structured statement is a SQL DROP statement, and call the index deletion interface of the ES cluster according to the index name in the SQLDROP statement to delete the corresponding index in the ES cluster; the third parsing module is used to parse the index name in the SQL TRUNCATE statement when the structured statement is a SQL TRUNCATE statement, build a corresponding document query according to the index name in the SQL TRUNCATE statement, call the document deletion interface of the ES cluster and delete the SQL based on the document query. All documents corresponding to the index name in the TRUNCATE statement. The above steps convert SQL DDL operations into Elasticsearch index management operations through the RESTful API, achieving operational standardization across different database systems.

[0117] In some embodiments, the language type of the structured statement is the data definition language, and the execution unit includes a fourth parsing module and a creation module. The fourth parsing module is used to parse the changed field content in the SQL ALTER statement when the structured statement is an SQL ALTER statement and is a field update, construct a JSON structure of the update mapping corresponding to the changed field content, and call the mapping update interface of the ES cluster through the PUT method to update the fields in the ES cluster according to the JSON structure of the update mapping; the creation module is used to use the Reindex API to create a new index when the structured statement is an SQL ALTER statement and is a field attribute update, and then convert the old index data into the new index data according to the content of the SQL ALTER statement and re-index the new index data into the new index. By converting the SQL ALTER statement into the mapping update or data re-indexing operation of Elasticsearch, the flexibility of modifying fields and data attributes is enhanced.

[0118] In some embodiments, the language type of the structured statement is the data manipulation language, and the execution unit includes a first extraction module, a second extraction module, and a third extraction module. The first extraction module is used to extract the document content and target index name in the SQL INSERT statement when the structured statement is a SQL INSERT statement, convert the document content in the SQL INSERT statement into JSON format to obtain the target document content, and use the POST method to call the INDEX API of the ES cluster to insert the target index name and the target document content into the ES cluster; the second extraction module is used to extract the update field, update value, and target document ID in the SQL UPDATE statement when the structured statement is a SQL UPDATE statement, construct a JSON structure including an update script based on the update field, the update value, and the target document ID, and call the UPDATE API of the ES cluster through the POST method to update the update script to the corresponding target document in the ES cluster, wherein the update script includes the update field and the update value; the third extraction module is used to extract the target document ID and index name in the SQL DELETE statement when the structured statement is a SQL DELETE statement, and call the DELETE of the ES cluster. API, and delete the corresponding target document from the ES cluster according to the target document ID and index name in the SQL DELETE statement. Allow users to use familiar SQL DML statements to operate the Elasticsearch cluster, maintaining operational consistency and user-friendliness.

[0119] In some embodiments, the language type of the structured statement is the data control language, and the execution unit includes a fourth extraction module, a fifth parsing module and a sixth parsing module. The fourth extraction module is used to extract the target user and the corresponding user permission information in the SQL GRANT statement when the structured statement is a SQLGRANT statement, and use the role management API to build or update the user information and the corresponding user permission information of the target user in the ES cluster according to the user permission information; the fifth parsing module is used to parse the new user name and the corresponding user name, password and user permissions in the CREATE USER statement when the structured statement is a CREATE USER statement, use the create user endpoint of the user management API and create a new user according to the user endpoint, the new user name in the CREATE USER statement and the corresponding user name, password and user permissions; the sixth parsing module is used to parse the DROP USER statement to obtain the deleted user name when the structured statement is a DROP USER statement, and call the delete user endpoint of the user management API to delete the user corresponding to the deleted user name. By converting the GRANT, CREATE USER, and DROP USER statements in SQL into Elasticsearch's RESTful API calls, a standardized operation interface is provided for user permission management, allowing SQL users to manage permissions and users without having to learn Elasticsearch-specific APIs.

[0120] In some embodiments, the language type of the structured statement is the data query language, and the execution unit includes an execution module and a seventh parsing module. The execution module is used to execute the structured statement in a preset processing method when the version of the ES cluster is greater than the preset version; the seventh parsing module is used to parse the structured statement to obtain extracted information when the version of the ES cluster is less than or equal to the preset version, construct the query DSL of the ES cluster based on the extracted information, and use the GET method to call the search API of the ES cluster, based on the search API, pass the constructed query DSL to query the data in the ES cluster to obtain the JSON response returned by the ES cluster, parse the JSON response, extract and format the query results, and the query DSL includes: return fields and query conditions. Regardless of the ES cluster version, users can use a unified SQL syntax to query data, which greatly improves the user-friendliness of the system and reduces the learning curve and operation complexity.

[0121] The ES cluster-based structured statement execution device includes a processor and memory. The acquisition unit and other components are stored as program units in the memory, and the processor executes the program units stored in the memory to implement the corresponding functions. The modules are all located in the same processor; alternatively, the modules can be located in different processors in any combination.

[0122] The processor contains a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be configured, and kernel parameters can be adjusted to address the existing problem of requiring operators to learn and master ElasticSearch's unique query language, which increases operational difficulty and labor costs.

[0123] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0124] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is run, the device where the computer-readable storage medium is located is controlled to execute the method for executing structured statements based on the ES cluster.

[0125] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the ES cluster-based structured statement execution method when the program is running.

[0126] An embodiment of the present invention provides a device, comprising a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are performed:

[0127] Step S201: obtaining a structured statement and verifying the grammatical correctness of the structured statement, wherein the language type of the structured statement is one of the following: data definition language, data manipulation language, data control language, and data query language;

[0128] Step S202: If the syntax of the structured statement is correct, parse the structured statement using SQL Parser to obtain a hierarchical abstract syntax tree.

[0129] Step S203, determining the language type of the structured statement according to the hierarchical abstract syntax tree;

[0130] Step S204: Process the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

[0131] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0132] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program for initializing at least the following method steps:

[0133] Step S201: obtaining a structured statement and verifying the grammatical correctness of the structured statement, wherein the language type of the structured statement is one of the following: data definition language, data manipulation language, data control language, and data query language;

[0134] Step S202: If the syntax of the structured statement is correct, parse the structured statement using SQL Parser to obtain a hierarchical abstract syntax tree.

[0135] Step S203, determining the language type of the structured statement according to the hierarchical abstract syntax tree;

[0136] Step S204: Process the structured statement using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

[0137] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0138] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0139] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0142] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0143] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0144] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0145] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0146] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0147] 1) The above-mentioned execution method of structured statements based on ES cluster in this application first obtains the structured statement and verifies the grammatical correctness of the structured statement. The language type of the structured statement is one of the following: data definition language, data manipulation language, data control language and data query language; then, when the grammar of the structured statement is correct, SQLParser is used to parse the structured statement to obtain a hierarchical abstract syntax tree; according to the hierarchical abstract syntax tree, the language type of the structured statement is determined; finally, the structured statement is processed using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and the target statement is executed to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement. This method focuses on bridging the compatibility gap between ElasticSearch and the SQL protocol, especially the support shortcomings for DDL, DCL and DML operations, and accurately converts SQL instructions into a domain-specific language (DSL) that adapts to the internal working principles of ElasticSearch. It reduces the cost for SQL users to become familiar with ElasticSearch's native API and syntax, simplifies the user operation process, and achieves deep integration of the SQL protocol and ElasticSearch. It solves the problem in existing technologies that require operators to learn and master ElasticSearch's unique query language, which increases operational difficulty and labor costs.

[0148] 2) The above-mentioned execution device of structured statements based on ES cluster of the present application includes an acquisition unit, a parsing unit, a determination unit and an execution unit. The acquisition unit is used to acquire structured statements and verify the grammatical correctness of the structured statements. The language type of the structured statements is one of the following: data definition language, data manipulation language, data control language and data query language; the parsing unit is used to parse the structured statements using SQL Parser when the grammar of the structured statements is correct to obtain a hierarchical abstract syntax tree; the determination unit is used to determine the language type of the structured statements according to the hierarchical abstract syntax tree; the execution unit is used to process the structured statements using a corresponding processing method according to the language type of the structured statements to convert the format of the structured statements into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds to the language type of the structured statement. The device focuses on bridging the compatibility gap between ElasticSearch and the SQL protocol, especially the support shortcomings for DDL, DCL and DML operations, and accurately converts SQL instructions into a domain-specific language (DSL) that adapts to the internal working principles of ElasticSearch. It reduces the cost for SQL users to become familiar with ElasticSearch's native API and syntax, simplifies the user operation process, and achieves deep integration of the SQL protocol and ElasticSearch. It solves the problem in existing technologies that require operators to learn and master ElasticSearch's unique query language, which increases operational difficulty and labor costs.

[0149] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for executing structured statements based on an ES cluster, characterized in that: include: Acquire a structured statement and verify the grammatical correctness of the structured statement, wherein the language type of the structured statement is one of the following: data definition language, data manipulation language, data control language, and data query language; When the syntax of the structured statement is correct, the structured statement is parsed using SQL Parser to obtain a hierarchical abstract syntax tree; Determining the language type of the structured statement according to the hierarchical abstract syntax tree; The structured statement is processed using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement, and the target statement is executed to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

2. The method according to claim 1, characterized in that The structured statement is parsed using SQL Parser to obtain a hierarchical abstract syntax tree, including: Based on the SQL Parser, a Lexer is used to read the structured statement, and multiple continuous texts in the structured statement are identified to obtain a token corresponding to each of the continuous texts, each of the continuous texts having corresponding features; Using a deterministic finite automaton and a predefined lexical file to determine the value and type of each token; All the tokens are input into a syntax analyzer, and a hierarchical abstract syntax tree is constructed using the syntax description of the structured statement as a rule, wherein a leaf node of the hierarchical abstract syntax tree is a part of a sub-statement in the structured statement.

3. The method according to claim 1, characterized in that The language type of the structured statement is the data definition language. The structured statement is processed using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement. The target statement is then executed to perform a corresponding operation on the ES cluster, including: In the case where the structured statement is an SQL CREATE statement, the index name and field definition in the SQL CREATE statement are parsed, and a corresponding creation mapping JSON structure is constructed according to the index name and field definition in the SQL CREATE statement. Thereafter, the index creation interface of the ES cluster is called using the PUT method to create an index in the ES cluster according to the creation mapping JSON structure. The creation mapping is used to define the document field structure and attributes in the ES cluster; In the case where the structured statement is an SQL DROP statement, parsing the index name in the SQL DROP statement, and calling the index deletion interface of the ES cluster according to the index name in the SQL DROP statement to delete the corresponding index in the ES cluster; In the case where the structured statement is an SQL TRUNCATE statement, the index name in the SQL TRUNCATE statement is parsed, a corresponding document query is constructed according to the index name in the SQL TRUNCATE statement, the document deletion interface of the ES cluster is called, and all documents corresponding to the index name in the SQL TRUNCATE statement are deleted based on the document query.

4. The method according to claim 1, wherein The language type of the structured statement is the data definition language. The structured statement is processed using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement. The target statement is then executed to perform a corresponding operation on the ES cluster, including: If the structured statement is an SQL ALTER statement and is a field update, parse the changed field content in the SQL ALTER statement, construct an update mapping JSON structure corresponding to the changed field content, and call the mapping update interface of the ES cluster through the PUT method to update the field in the ES cluster according to the update mapping JSON structure; In the case where the structured statement is an SQL ALTER statement and is a field attribute update, a new index is created using the Reindex API, and then old index data is converted into new index data according to the content of the SQL ALTER statement, and the new index data is re-indexed into the new index.

5. The method according to claim 1, characterized in that The language type of the structured statement is the data manipulation language. The structured statement is processed using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement. The target statement is then executed to perform a corresponding operation on the ES cluster, including: In the case where the structured statement is an SQL INSERT statement, extracting the document content and the target index name in the SQL INSERT statement, converting the document content in the SQL INSERT statement into JSON format to obtain the target document content, and calling the INDEX API of the ES cluster using the POST method to insert the target index name and the target document content into the ES cluster; In the case where the structured statement is an SQL UPDATE statement, extracting the update fields, update values, and target document ID in the SQL UPDATE statement, constructing a JSON structure including an update script based on the update fields, the update values, and the target document ID, and calling the UPDATE API of the ES cluster through the POST method to update the update script to the corresponding target document in the ES cluster, where the update script includes the update fields and the update values; In the case where the structured statement is an SQL DELETE statement, the target document ID and index name in the SQL DELETE statement are extracted, the DELETE API of the ES cluster is called, and the corresponding target document is deleted from the ES cluster according to the target document ID and index name in the SQL DELETE statement.

6. The method according to claim 1, characterized in that The language type of the structured statement is the data control language. The structured statement is processed using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement. The target statement is then executed to perform a corresponding operation on the ES cluster, including: In the case where the structured statement is an SQL GRANT statement, extract the target user and corresponding user permission information in the SQL GRANT statement, and use the role management API to construct or update the user information and corresponding user permission information of the target user in the ES cluster according to the user permission information; In a case where the structured statement is a CREATE USER statement, parsing a new user name, a corresponding user name, a password, and user permissions in the CREATE USER statement, and creating a new user using a create user endpoint of a user management API based on the user endpoint, the new user name, the corresponding user name, a password, and user permissions in the CREATE USER statement; In the case where the structured statement is a DROP USER statement, the DROP USER statement is parsed to obtain a deleted user name, and a delete user endpoint of the user management API is called to delete the user corresponding to the deleted user name.

7. The method according to claim 1, characterized in that The language type of the structured statement is the data query language. The structured statement is processed using a corresponding processing method according to the language type of the structured statement to convert the format of the structured statement into a target format to obtain a target statement. The target statement is then executed to perform corresponding operations on the ES cluster, including: If the version of the ES cluster is greater than the preset version, the structured statement is executed in a preset processing mode; When the version of the ES cluster is less than or equal to the preset version, the structured statement is parsed to obtain extracted information, the query DSL of the ES cluster is constructed based on the extracted information, and the search API of the ES cluster is called using the GET method. The constructed query DSL is passed in based on the search API to query the data in the ES cluster to obtain a JSON response returned by the ES cluster, the JSON response is parsed, and the query results are extracted and formatted. The query DSL includes: return fields and query conditions.

8. A structured statement execution device based on ES cluster, characterized in that: include: an acquisition unit, configured to acquire a structured statement and verify the grammatical correctness of the structured statement, wherein the language type of the structured statement is one of the following: a data definition language, a data manipulation language, a data control language, and a data query language; A parsing unit, configured to parse the structured statement using SQL Parser to obtain a hierarchical abstract syntax tree when the syntax of the structured statement is correct; a determining unit, configured to determine a language type of the structured statement according to the hierarchical abstract syntax tree; An execution unit is used to process the structured statement using a corresponding processing method according to the language type of the structured statement, so as to convert the format of the structured statement into a target format to obtain a target statement, and execute the target statement to perform corresponding operations on the ES cluster. The target format is a format that can be recognized by the ES cluster, and the processing method corresponds one-to-one to the language type of the structured statement.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for executing structured statements based on the ES cluster according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include an execution method for executing the ES cluster-based structured statements described in any one of claims 1 to 7.

Images