Data search method, system and equipment and storage medium

Abstract

The invention discloses a data search method, system and device and a storage medium. The method comprises the steps of obtaining a lexical file and a grammar file customized by an administrator; wherein the lexical file is used for defining keywords and characters used in the SQL, and the grammar file is used for defining a grammar use mode in the SQL; analyzing the lexical file and the grammar file through an Antrl tool to generate a corresponding grammar tree file; generating a corresponding execution logic plan according to the syntax tree file, and performing association configuration on the execution logic plan and Elasticsearch; and receiving a search request of a user for data in Elasticsearch, responding to the search request according to the execution logic plan, and feeding back a search result. According to the method, the user can be helped to realize Elasticsearch data search based on the SQL statement, and the search accuracy is relatively high. The method can be widely applied to the technical field of data search.

Description

Technical Field

[0001] The present application relates to the field of data search technology, and in particular to a data search method, system, device and storage medium. Background Art

[0002] At present, with the development of information technology, more and more application services have emerged, bringing more convenience and fresh experience to people's work and life. Among them, with the advent of the big data era, the storage, retrieval and analysis of massive data have become important challenges faced by enterprises. As the most popular open source distributed search engine, Elasticsearch (ES) has been widely used in log analysis, full-text search and other fields with its efficient full-text search capabilities and scalability. However, the query language DSL (Domain Specific Language) natively supported by Elasticsearch has a high complexity and learning curve for non-technical users, which limits its use in a wider range of scenarios. At the same time, SQL, as a standard language for database queries, has a broad application base in the field of data queries because of its intuitiveness, easy understanding and use. Therefore, converting the SQL query language into Elasticsearch DSL and realizing SQL search on Elasticsearch has become an important direction to improve the usability of Elasticsearch and expand application scenarios.

[0003] In the related art, when implementing the conversion from SQL to Elasticsearch DSL, most methods have problems such as low parsing efficiency, poor compatibility, and lack of support for complex queries. Especially when dealing with complex SQL queries, such as those containing nested queries, aggregate functions, and multi-table associations, existing conversion methods are often difficult to accurately and efficiently convert into DSL query statements supported by Elasticsearch, resulting in poor search results and a bad user experience.

[0004] In summary, the problems existing in related technologies need to be solved urgently. Summary of the invention

[0005] The purpose of this application is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, an object of embodiments of the present application is to provide a data search method, system, device and storage medium.

[0007] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present application include:

[0008] On the one hand, an embodiment of the present application provides a data search method, which is applied to a playback terminal, and the method includes:

[0009] Obtain the administrator-defined lexical file and grammar file; wherein the lexical file is used to define the keywords and characters used in SQL, and the grammar file is used to define the syntax usage in SQL;

[0010] Parsing the lexical file and the grammar file by using the Antrl tool to generate a corresponding grammar tree file;

[0011] Generate a corresponding execution logic plan according to the syntax tree file, and associate the execution logic plan with Elasticsearch;

[0012] Receive a user's search request for data in the Elasticsearch, respond to the search request according to the execution logic plan, and feed back search results.

[0013] In addition, the data search method according to the above embodiment of the present application may also have the following additional technical features:

[0014] Further, in one embodiment of the present application, the lexical file and the grammar file are parsed by the Antrl tool to generate a corresponding grammar tree file, including:

[0015] Generate a lexical analyzer and a syntax analyzer through the Antrl tool;

[0016] Parsing the lexical file based on the lexical analyzer, converting the lexical rules in the lexical file into a token sequence;

[0017] The tag sequence is input into the syntax analyzer, the syntax file is parsed based on the syntax analyzer, and the tag sequence is processed according to the syntax rules obtained by parsing to obtain a corresponding syntax tree file.

[0018] Furthermore, in one embodiment of the present application, the method further includes:

[0019] Obtaining identification information input by the administrator; the identification information is used to specify the Elasticsearch corresponding to the lexical file and the grammar file;

[0020] The configuration of associating the execution logic plan with Elasticsearch includes:

[0021] The execution logic plan is associated with the Elasticsearch corresponding to the identification information and configured.

[0022] Furthermore, in one embodiment of the present application, before the step of receiving a user's search request for data in the Elasticsearch, the method further includes:

[0023] Building a high-level API corresponding to the Elasticsearch; wherein the high-level API includes a RestHighLevelClient object;

[0024] A connection is established between the high-level API and the Elasticsearch.

[0025] Further, in one embodiment of the present application, the building of a high-level API corresponding to the Elasticsearch and establishing a connection with the Elasticsearch through the high-level API includes:

[0026] Obtain the cluster information of the Elasticsearch;

[0027] Create a RestClient object through the constructor mode, and pass the cluster information to the RestClient object;

[0028] Create a RestHighLevelClient object, and pass the RestClient object to the RestHighLevelClient object;

[0029] A connection is established between the RestHighLevelClient object and the Elasticsearch.

[0030] Furthermore, in one embodiment of the present application, the method further includes:

[0031] According to the cluster information, determine whether a user password is set in the Elasticsearch cluster;

[0032] If a user password is set in the Elasticsearch cluster, create a BasicCredentialsProvider object, set the user password in the BasicCredentialsProvider object, and pass the BasicCredentialsProvider object to the RestClient object.

[0033] Furthermore, in one embodiment of the present application, the method further includes:

[0034] Create a database connection pool;

[0035] According to the database connection pool, the connection of the Elasticsearch is optimized.

[0036] On the other hand, an embodiment of the present application provides a data search system, the system comprising:

[0037] An acquisition unit is used to acquire a lexical file and a grammar file customized by an administrator; wherein the lexical file is used to define keywords and characters used in SQL, and the grammar file is used to define the usage of grammar in SQL;

[0038] A parsing unit, used to parse the lexical file and the grammar file through an Antrl tool to generate a corresponding grammar tree file;

[0039] A processing unit, configured to generate a corresponding execution logic plan according to the syntax tree file, and associate the execution logic plan with Elasticsearch;

[0040] The response unit is used to receive a user's search request for data in the Elasticsearch, respond to the search request according to the execution logic plan, and feed back search results.

[0041] On the other hand, an embodiment of the present application provides an electronic device, including:

[0042] at least one processor;

[0043] at least one memory for storing at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the above-mentioned data search method.

[0045] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned data search method.

[0046] The advantages and benefits of the present application will be partially given in the following description, and partially become apparent from the following description, or be understood through the practice of the present application:

[0047] The data search method, system, device and storage medium disclosed in the embodiment of the present application obtain the administrator's customized lexical file and grammar file; wherein the lexical file is used to define the keywords and characters used in SQL, and the grammar file is used to define the grammar usage in SQL; the lexical file and the grammar file are parsed by the Antrl tool to generate a corresponding grammar tree file; a corresponding execution logic plan is generated according to the grammar tree file, and the execution logic plan is associated with Elasticsearch for configuration; a user's search request for data in the Elasticsearch is received, the search request is responded to according to the execution logic plan, and the search result is fed back. This method can help users realize Elasticsearch data search based on SQL statements, and the search accuracy is high, which is conducive to improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present application. For technical personnel in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of an implementation environment of a data search method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of a data search method provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of a process for generating a syntax tree file provided in an embodiment of the present application;

[0052] Figure 4 A diagrammatic diagram of an abstract syntax tree provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of the structure of an abstract syntax tree provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of an execution logic plan provided in an embodiment of the present application;

[0055] Figure 7 A flowchart of a high-level API provided in an embodiment of the present application;

[0056] Figure 8A Druid connection pool diagram of Elasticsearch provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of the structure of a data search system provided in an embodiment of the present application;

[0058] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0060] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0062] 1)Antrl (Another Tool for Language Recognition): A powerful parser generation tool that automatically creates lexical analyzers and syntax analyzers based on user-defined grammar rules. It supports multiple programming languages, such as Java, C++, etc., and is widely used in compiler construction, language processing, and complex text parsing scenarios, simplifying the development process of these tasks and improving development efficiency.

[0063] 2) SQL (Structured Query Language): A programming language for operating relational databases. It is used to access data, query, update, and manage relational database systems, and has functions such as data definition, data manipulation, and data control. SQL statements such as SELECT, INSERT, UPDATE, DELETE, etc. can be used to query, insert, update, and delete data in the database. SQL is concise, easy to learn, and easy to use, and is the standard language for database management and operation.

[0064] 3) Elasticsearch: An open source distributed search and analysis engine, it is built on Lucene and provides efficient full-text search and data analysis capabilities. Elasticsearch supports multiple data formats and has features such as distributed architecture, high availability, and scalability. It is widely used in search engines, log analysis, data warehouses, and other scenarios, providing enterprises with powerful data search and analysis solutions.

[0065] 4) Druid: A high-performance database connection pool open sourced by Alibaba, with rich monitoring and statistics functions, which can significantly improve database operation performance. It supports advanced features such as connection pool preheating, SQL injection protection, and database password encryption, and has a built-in powerful StatFilter plug-in that can provide detailed statistics on SQL execution performance to help developers optimize database access. The Druid connection pool also provides flexible configuration options, suitable for a variety of complex database application scenarios.

[0066] 5) Abstract Syntax Tree (AST): A tree representation of the abstract syntax structure of source code. It discards redundant information in the source code that does not affect the program logic (such as spaces, line breaks, comments, etc.), and only retains elements that are critical to the program structure and meaning. AST is an important component of compilers and interpreters for further analysis, optimization, and code generation. Through AST, developers can more easily understand and manipulate the structure of program code.

[0067] 6) Execution logic plan refers to a series of steps and operation processes defined when interacting with Elasticsearch. It is a detailed execution logic plan that covers all stages from environment preparation to specific operations.

[0068] 7) High-Level API, an API interface that simplifies the underlying details and provides a higher level of abstraction. In Elasticsearch, the high-level API usually refers to `RestHighLevelClient`, which is a Java client library designed specifically for Elasticsearch. It provides a more friendly and easy-to-use API interface, allowing developers to interact with Elasticsearch more conveniently.

[0069] At present, with the development of information technology, more and more application services have emerged, bringing more convenience and fresh experience to people's work and life. Among them, with the advent of the big data era, the storage, retrieval and analysis of massive data have become important challenges faced by enterprises. As the most popular open source distributed search engine, Elasticsearch (ES) has been widely used in log analysis, full-text search and other fields with its efficient full-text search capabilities and scalability. However, the query language DSL (Domain Specific Language) natively supported by Elasticsearch has a high complexity and learning curve for non-technical users, which limits its use in a wider range of scenarios. At the same time, SQL, as a standard language for database queries, has a broad application base in the field of data queries because of its intuitiveness, easy understanding and use. Therefore, converting the SQL query language into Elasticsearch DSL and realizing SQL search on Elasticsearch has become an important direction to improve the usability of Elasticsearch and expand application scenarios.

[0070] In the related art, when implementing the conversion from SQL to Elasticsearch DSL, most methods have problems such as low parsing efficiency, poor compatibility, and lack of support for complex queries. Especially when dealing with complex SQL queries, such as those containing nested queries, aggregate functions, and multi-table associations, existing conversion methods are often difficult to accurately and efficiently convert into DSL query statements supported by Elasticsearch, resulting in poor search results and a bad user experience.

[0071] In view of this, a data search method is provided in an embodiment of the present application, which obtains a lexical file and a grammar file customized by an administrator; wherein the lexical file is used to define the keywords and characters used in SQL, and the grammar file is used to define the usage of the grammar in SQL; the lexical file and the grammar file are parsed by the Antrl tool to generate a corresponding grammar tree file; a corresponding execution logic plan is generated according to the grammar tree file, and the execution logic plan is associated with Elasticsearch and configured; a user's search request for data in the Elasticsearch is received, the search request is responded to according to the execution logic plan, and the search result is fed back. This method can help users realize Elasticsearch data search based on SQL statements, and the search accuracy is high, which is conducive to improving user experience.

[0072] Please refer to Figure 1 , Figure 1A schematic diagram of an implementation environment of a data search method provided in an embodiment of the present application is shown. In the implementation environment, the main hardware and software entities involved include a terminal device 110 and a backend server 120. The terminal device 110 and the backend server 120 are connected in communication. The terminal device 110 can be used for a user to initiate a data search request, and the backend server 120 can be used to store related data resources, respond to the user's data search request, and feedback search results to the user.

[0073] Specifically, the data search method provided in the embodiments of the present application can be configured on the backend server 120 side and executed based on the data interaction between the terminal device 110 and the backend server 120. Among them, the terminal device 110 of the above embodiment may include a mobile phone, a computer, a smart wearable device, an intelligent voice interaction device, a vehicle terminal, etc., but is not limited to this. The backend server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0074] The terminal device 110 and the backend server 120 may establish a communication connection via a wireless network or a wired network. The wireless network or wired network uses standard communication technology and / or protocols, and the network may be set to the Internet or any other network, such as but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.

[0075] Of course, it is understandable that Figure 1 The implementation environment in the embodiment of the present application is only some optional application scenarios of the data search method provided in the embodiment of the present application, and the actual application is not fixed. Figure 1 The hardware and software environment shown.

[0076] Below, in combination with the introduction of the aforementioned implementation environment, a data search method provided in an embodiment of the present application is introduced and explained.

[0077] Please refer to Figure 2 , Figure 2is a schematic diagram of a data search method provided in an embodiment of the present application. The data search method can be applied to a playback terminal. Specifically, the data search method provided in an embodiment of the present application includes but is not limited to:

[0078] Step 210: Obtain the lexical file and grammar file customized by the administrator; wherein the lexical file is used to define the keywords and characters used in SQL, and the grammar file is used to define the usage of the grammar in SQL;

[0079] Step 220: Parse the lexical file and the grammar file by using the Antrl tool to generate a corresponding grammar tree file;

[0080] Step 230: Generate a corresponding execution logic plan according to the syntax tree file, and associate the execution logic plan with Elasticsearch;

[0081] Step 240: Receive a user's search request for data in the Elasticsearch, respond to the search request according to the execution logic plan, and feed back search results.

[0082] In an embodiment of the present application, a data search method is provided. The method parses SQL statements based on the Antrl tool and converts them into Elasticsearch DSL query statements. The method can accurately parse various query conditions, aggregation functions, multi-table associations and other complex structures in the SQL statements, and generate corresponding Elasticsearch DSL query statements, thereby realizing efficient retrieval of SQL on Elasticsearch. The search accuracy is high, which is conducive to improving the user experience.

[0083] Specifically, the embodiment of the present application aims to achieve seamless connection between SQL (Structured Query Language) and Elasticsearch (a distributed search engine) by integrating advanced language processing technology and efficient data retrieval mechanism. When implementing the method provided in the embodiment of the present application, first, the administrator-defined lexical file and grammar file can be obtained. Here, the administrator can be the management and maintenance personnel of the application or system that provides data search services. The administrator can set and change the functions related to data search. In the embodiment of the present application, there is no restriction on their specific identity. In the embodiment of the present application, the administrator-defined lexical file can be used to define the keywords and characters used in SQL. The types of keywords here can include multiple types, such as `SELECT`, `FROM`, `WHERE`, `JOIN`, etc. Similarly, the types of characters can also include multiple types, such as brackets `()`, semicolons `;`, equal signs `=`, etc. Exemplarily, the following contents may be included in a lexical file:

[0084] SEMICOLON:';';

[0085] LEFT_PAREN:'(';

[0086] RIGHT_PAREN:')';

[0087] COMMA:',';

[0088] DOT:'.';

[0089] ALTER:'ALTER';

[0090] SELECT:'SELECT';

[0091] FROM:'FROM';

[0092] WHERE:'WHERE';

[0093] TABLE:'TABLE';

[0094] EQ:'='|'==';

[0095] NSEQ:'<=>';

[0096] NEQ:'<>';

[0097] NEQJ:'!=';

[0098] LT:'<';

[0099] LTE:'<='|'!>';

[0100] GT:'>';

[0101] GTE:'>='|'!<';

[0102] MATCH:'MATCH';

[0103] MATCH_PHRASE:'MATCH_PHRASE';

[0104] TERM:'TERM';

[0105] MULTI_MATCH:'MULTI_MATCH';

[0106] FUZZY:'FUZZY';

[0107] MISSING_QUERY:'MISSING_QUERY';

[0108] Of course, it can be understood that the above content is only used to exemplify the lexical files involved in the embodiments of the present application. The specific definition method can be flexibly set according to actual needs, and the present application does not impose any restrictions on this.

[0109] In the embodiment of the present application, a grammar file customized by the administrator is also obtained, and the grammar file can be used to define the usage of grammar in SQL. For example, the structure and grammar rules of SQL queries can be defined, such as how to combine keywords and characters to form a valid SQL query. In addition, in the embodiment of the present application, in the grammar file, the administrator can also add some special queries related to Elasticsearch, such as match, term, wildcard and other query methods. Exemplarily, a grammar file may include the following content:

[0110] parser grammar SqlBaseParser;

[0111] options{tokenVocab=SqlBaseLexer;}

[0112] @members{

[0113] public boolean legacy_setops_precedence_enabled=false;

[0114] public boolean legacy_exponent_literal_as_decimal_enabled=false;

[0115] public boolean SQL_standard_keyword_behavior=false;

[0116] public boolean double_quoted_identifiers=false;

[0117] }

[0118] singleStatement

[0119] :statement SEMICOLON*EOF;

[0120] singleFunctionIdentifier

[0121] :functionIdentifier EOF;

[0122] singleDataType

[0123] :dataType EOF;

[0124] singleTableSchema

[0125] :colTypeList EOF;

[0126] comparisonOperator

[0127] :EQ|NEQ|NEQJ|LT|LTE|GT|GTE|NSEQ|MATCH|MATCH_PHRASE|TERM|WILD_CARD

[0128] Of course, it can be understood that the above content is only used to exemplify the grammar file involved in the embodiments of the present application. The specific definition method can be flexibly set according to actual needs, and the present application does not impose any restrictions on this.

[0129] In the embodiment of the present application, after obtaining the administrator's customized lexical file and grammar file, the Antrl tool can be used to parse the lexical file and grammar file to generate the corresponding grammar tree file. The Antrl tool is a powerful parser generation tool that can automatically generate a parser based on grammar rules to parse SQL statements and convert them into Elasticsearch DSL query statements. Please refer to Figure 3 , Figure 3 A schematic diagram of a process for generating a syntax tree file provided in an embodiment of the present application is shown. In an embodiment of the present application, according to the lexical file and grammar file customized by the administrator, the Antrl tool can generate relevant lexical analyzers and grammar analyzers, parse the lexical file and grammar file, and generate corresponding syntax tree files. The syntax tree file here can be in Java language or other types of languages, and this application does not limit this. For example, please refer to Figure 4 and Figure 5 , Figure 4 A diagrammatic representation of an abstract syntax tree provided in an embodiment of the present application is shown. Figure 5 A schematic diagram of the structure of an abstract syntax tree provided in an embodiment of the present application is shown. Figure 5 The abstract syntax tree is constructed by taking the SQL statement SELECT F1 FROM T1 WHERE F1 MATCH'11' as an example. The `MATCH` function used in this SQL statement filters `F1`, which corresponds to the query method of the field MATCH in Elasticsearch.

[0130] In an embodiment of the present application, after obtaining the syntax tree file, a corresponding execution logic plan can be generated according to the syntax tree file, and it can be associated with Elasticsearch for configuration. Specifically, when generating the execution logic plan, the syntax tree can be traversed, and the corresponding execution logic plan can be generated according to the structure of the tree. For example, for a query such as `SELECT * FROM table WHERE condition`, an execution plan containing information such as the table name and selection conditions is generated. Then, the generated execution logic plan can be mapped to the indexes and fields in Elasticsearch, and these mapping relationships can be stored in a configuration file or database for use in subsequent queries. For example, please refer to Figure 6 , Figure 6 A schematic diagram of an execution logic plan provided in an embodiment of the present application is shown.

[0131] After the above database is established, it can be put into the data search service to respond to the user's data search request. Specifically, in the embodiment of the present application, when the user has a need for data search, a data search request for Elasticsearch can be submitted through the front-end interface or other means. After receiving the user's search request, it can be parsed into an SQL query statement. Then, the previously generated execution logic plan can be called according to the parsed SQL query statement, and the data in Elasticsearch can be queried according to the execution logic plan, and the search results are fed back to the user.

[0132] It should be noted that in the embodiment of the present application, if the input does not conform to any defined rules, an error prompt may be given. The administrator may customize the error handling logic to enhance robustness, and the present application does not limit this.

[0133] It can be understood that a data search method is provided in the embodiment of the present application, which obtains a lexical file and a grammar file customized by an administrator; wherein the lexical file is used to define the keywords and characters used in SQL, and the grammar file is used to define the usage of the grammar in SQL; the lexical file and the grammar file are parsed by the Antrl tool to generate a corresponding grammar tree file; a corresponding execution logic plan is generated according to the grammar tree file, and the execution logic plan is associated with Elasticsearch and configured; a user's search request for data in the Elasticsearch is received, the search request is responded to according to the execution logic plan, and the search results are fed back. This method solves the problems of low parsing efficiency, poor compatibility, and lack of support for complex queries in the conversion process from SQL to Elasticsearch DSL in the prior art, and provides strong support for the application of Elasticsearch in a wider range of scenarios. Therefore, the present application has important application value and broad market prospects in the fields of big data processing and full-text search.

[0134] Specifically, in some embodiments, the parsing of the lexical file and the grammar file by the Antrl tool to generate a corresponding grammar tree file includes:

[0135] Generate a lexical analyzer and a syntax analyzer through the Antrl tool;

[0136] Parsing the lexical file based on the lexical analyzer, converting the lexical rules in the lexical file into a token sequence;

[0137] The tag sequence is input into the syntax analyzer, the syntax file is parsed based on the syntax analyzer, and the tag sequence is processed according to the syntax rules obtained by parsing to obtain a corresponding syntax tree file.

[0138] In the embodiment of the present application, when the syntax tree file is generated by the Antrl tool, in some embodiments, after receiving the lexical file and the grammar file, the corresponding parser class can be automatically generated. These classes include a lexical analyzer (Lexer) and a grammatical analyzer (Parser). The lexical analyzer is responsible for reading the original input text and converting it into independent tags based on the defined lexical rules. The grammatical analyzer works on the basis of the lexical analyzer. It receives the tag sequence generated by the lexical analyzer and attempts to construct an abstract syntax tree according to the defined grammatical rules. Once the abstract syntax tree is generated, the developer can write the code of the visitor pattern (Visitor Pattern) or the listener pattern (ListenerPattern) to traverse this tree and perform corresponding operations. For example, in a compiler, the abstract syntax tree can be traversed to generate target code or perform various static analyses.

[0139] Specifically, in some embodiments, the method further includes:

[0140] Obtaining identification information input by the administrator; the identification information is used to specify the Elasticsearch corresponding to the lexical file and the grammar file;

[0141] The configuration of associating the execution logic plan with Elasticsearch includes:

[0142] The execution logic plan is associated with the Elasticsearch corresponding to the identification information and configured.

[0143] In the embodiment of the present application, when uploading the lexical file and the grammar file, the administrator can also upload the identification information synchronously, and the identification information can be used to indicate the Elasticsearch corresponding to the lexical file and the grammar file. When the execution logic plan is subsequently generated, it can be associated with the Elasticsearch corresponding to the identification information.

[0144] Specifically, in some embodiments, before the step of receiving a user's search request for data in the Elasticsearch, the method further includes:

[0145] Building a high-level API corresponding to the Elasticsearch; wherein the high-level API includes a RestHighLevelClient object;

[0146] A connection is established between the high-level API and the Elasticsearch.

[0147] In the embodiment of the present application, Elasticsearch provides client libraries in multiple languages. When using them, you can use high-level APIs to establish them. For example, you can use RestHighLevelClient, which is a Java library for interacting with Elasticsearch in a higher-level way. Compared with low-level APIs, high-level APIs provide richer functions and better usability, which can significantly improve the stability and performance of the system.

[0148] Specifically, in some embodiments, building a high-level API corresponding to the Elasticsearch and establishing a connection with the Elasticsearch through the high-level API includes:

[0149] Obtain the cluster information of the Elasticsearch;

[0150] Create a RestClient object through the constructor mode, and pass the cluster information to the RestClient object;

[0151] Create a RestHighLevelClient object, and pass the RestClient object to the RestHighLevelClient object;

[0152] A connection is established between the RestHighLevelClient object and the Elasticsearch.

[0153] In an embodiment of the present application, when establishing a connection with Elasticsearch through RestHighLevelClient, you can first obtain the cluster information of Elasticsearch, such as cluster node information and cluster connection information, in a format such as `ip:port`, where `ip` is the ip of the Elasticsearch cluster node, and `port` is the http port of the Elasticsearch cluster node. Multiple node information is stored in an array. Then, you can create a RestClient object through the constructor pattern and pass the cluster information to the RestClient object. After the RestClient object is established, you can create a RestHighLevelClient object and pass the established RestClient object to the RestHighLevelClient object, so that you can establish a connection with Elasticsearch through the RestHighLevelClient object.

[0154] Reference Figure 7 , Figure 7A flowchart of using a high-level API provided in an embodiment of the present application is shown. Specifically, in some embodiments, the method further includes:

[0155] According to the cluster information, determine whether a user password is set in the Elasticsearch cluster;

[0156] If the Elasticsearch is set with a user password, create a BasicCredentialsProvider object, set the user password in the BasicCredentialsProvider object, and pass the BasicCredentialsProvider object to the RestClient object.

[0157] In the embodiment of the present application, in some cases, it is also possible to determine whether a user password is set in the Elasticsearch cluster based on the cluster information. If so, a BasicCredentialsProvider object can be created, the user password can be set in it, and then passed to the RestClient object, thereby facilitating the establishment of a connection with Elasticsearch.

[0158] Specifically, in some embodiments, the method further includes:

[0159] Create a database connection pool;

[0160] According to the database connection pool, the connection of the Elasticsearch is optimized.

[0161] In an embodiment of the present application, in order to cope with request processing in high-concurrency scenarios, a database connection pool may also be created, and the connection to Elasticsearch may be optimized based on the database connection pool. By managing the connection of the Elasticsearch client through the connection pool, effective reuse of connection resources may be achieved, and the concurrent processing capability and response speed of the system may be improved. Exemplarily, in an embodiment of the present application, a database connection pool may be created based on Druid, and DruidPooledConnection, DruidDataSourceFactory, DruidDataSource, and DruidPooledPreparedStatement classes of Druid may be rewritten, wherein the connection to the Elasticsearch cluster is implemented, and parameters for the Elasticsearch connection are added to optimize the Elasticsearch connection. For example, referring to Figure 8 , Figure 8A Druid connection pool diagram of Elasticsearch provided in an embodiment of the present application is shown.

[0162] It can be understood that the data search method in the embodiment of the present application parses the SQL statement of querying Elasticsearch by using the Antrl syntax analysis tool, and has the same lexical and grammatical parsing capabilities as SparkSQL, and customizes the syntax to implement special queries of Elasticsearch, such as match, term, wildcard and other query methods, and can customize the lexical and grammar for different application scenarios. Therefore, the SQL syntax can be more complex and flexible, and can improve the ease of use and query capabilities of SQL.

[0163] In addition, when connecting to Elasticsearch, the RestHighLevelClient method is used, which has higher version compatibility than the TransportClient connection method, and transmits data through SSL / TLS encryption, making it safer and more reliable to use. At the same time, Druid is used for data connection pool management, which can significantly reduce overhead and respond quickly to frequent queries. In high-concurrency scenarios, the connection pool can evenly allocate database connection resources to avoid performance bottlenecks caused by excessive use of a certain connection.

[0164] The technical solution of this application has at least the following advantages:

[0165] 1) Improve usability: SQL is a widely used data query language, which is much more intuitive and understandable than Elasticsearch's native query DSL. The primary purpose of this application is to query Elasticsearch through SQL, lowering the threshold for non-technical users and enabling more people to easily use Elasticsearch's powerful search capabilities.

[0166] 2) Enhanced query capabilities: Existing SQL to Elasticsearch conversion tools often seem incapable of processing complex queries, such as nested queries, multi-table joins, complex aggregations, etc. This application introduces the ANTLR (Another Tool for Language Recognition) syntax parser, which can accurately parse complex SQL statements and generate corresponding Elasticsearch DSL queries, thereby significantly improving the query capability and flexibility of the system.

[0167] 3) Improve performance and compatibility: Using Elasticsearch's high-level API (such as RestHighLevelClient) for data transmission and request processing can effectively improve the efficiency of data retrieval and be compatible with different versions of Elaisticsearch. At the same time, by introducing database connection pool technology, this application realizes the intelligent management and effective reuse of Elasticsearch client connections, effectively responds to high-concurrency requests, and improves the stability and response speed of the system.

[0168] Reference Fig. 9 In an embodiment of the present application, a data search system is also provided, the data search system comprising:

[0169] The acquisition unit 910 is used to acquire a lexical file and a grammar file customized by an administrator; wherein the lexical file is used to define keywords and characters used in SQL, and the grammar file is used to define the usage of grammar in SQL;

[0170] A parsing unit 920, used to parse the lexical file and the grammar file through an Antrl tool to generate a corresponding grammar tree file;

[0171] A processing unit 930 is used to generate a corresponding execution logic plan according to the syntax tree file, and associate the execution logic plan with Elasticsearch;

[0172] The response unit 940 is used to receive a user's search request for data in the Elasticsearch, respond to the search request according to the execution logic plan, and feed back search results.

[0173] Furthermore, in one embodiment of the present application, the parsing unit is specifically used for:

[0174] Generate a lexical analyzer and a syntax analyzer through the Antrl tool;

[0175] Parsing the lexical file based on the lexical analyzer, converting the lexical rules in the lexical file into a token sequence;

[0176] The tag sequence is input into the syntax analyzer, the syntax file is parsed based on the syntax analyzer, and the tag sequence is processed according to the syntax rules obtained by parsing to obtain a corresponding syntax tree file.

[0177] Furthermore, in one embodiment of the present application, the data search system further includes an association unit, and the association unit is specifically used to:

[0178] Obtaining identification information input by the administrator; the identification information is used to specify the Elasticsearch corresponding to the lexical file and the grammar file;

[0179] The execution logic plan is associated with the Elasticsearch corresponding to the identification information and configured.

[0180] Furthermore, in one embodiment of the present application, the data search system further includes a connection unit, and the connection unit is specifically used to:

[0181] Building a high-level API corresponding to the Elasticsearch; wherein the high-level API includes a RestHighLevelClient object;

[0182] A connection is established between the high-level API and the Elasticsearch.

[0183] Further, in one embodiment of the present application, the connecting unit is specifically used for:

[0184] Obtain the cluster information of the Elasticsearch;

[0185] Create a RestClient object through the constructor mode, and pass the cluster information to the RestClient object;

[0186] Create a RestHighLevelClient object, and pass the RestClient object to the RestHighLevelClient object;

[0187] A connection is established between the RestHighLevelClient object and the Elasticsearch.

[0188] Furthermore, in one embodiment of the present application, the connection unit is also used for:

[0189] According to the cluster information, determine whether a user password is set in the Elasticsearch cluster;

[0190] If a user password is set in the Elasticsearch cluster, create a BasicCredentialsProvider object, set the user password in the BasicCredentialsProvider object, and pass the BasicCredentialsProvider object to the RestClient object.

[0191] Furthermore, in one embodiment of the present application, the data search system further includes a concurrency optimization unit, and the concurrency optimization unit is specifically used to:

[0192] Create a database connection pool;

[0193] According to the database connection pool, the connection of the Elasticsearch is optimized.

[0194] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0195] The present application also provides an electronic device, including:

[0196] at least one processor;

[0197] at least one memory for storing at least one program;

[0198] When at least one program is executed by at least one processor, the at least one processor implements the above data search method.

[0199] The electronic device of the embodiment of the present application may be a terminal device, a computer device or a server device.

[0200] For example, taking the electronic device as a server device as an example, referring to Fig.10 The server device 1000 may have relatively large differences due to different configurations or performances, and may include one or more central processing units 1010 (abbreviated as CPU, Central Processing Units) and memory 1060, and one or more storage media 1030 (such as one or more mass storage devices) storing application programs 1033 or data 1032. Among them, the memory 1060 and the storage medium 1030 can be temporary storage or permanent storage. The program stored in the storage medium 1030 may include one or more units or modules, and each unit or module may include a series of operating instructions in the server device 1000. Furthermore, the central processing unit 1010 can be configured to communicate with the storage medium 1030 and execute a series of operating instructions in the storage medium 1030 on the server device 1000.

[0201] The server device 1000 may further include one or more power supplies 1020 , one or more wired or wireless network interfaces 1040 , one or more input and output interfaces 1050 , and one or more operating systems 1031 .

[0202] The CPU 1010 in the server device 1000 may be used to execute the following Figure 2 An embodiment of a data search method is shown.

[0203] Similarly, the contents of the above method embodiments are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0204] The embodiment of the present application further provides a computer-readable storage medium, in which a program executable by the central processing unit 1010 is stored. The program executable by the central processing unit 1010 is used to perform the above-mentioned data search method when executed by the central processing unit 1010.

[0205] Similarly, the contents of the above method embodiments are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0206] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the application is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0207] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present application. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional techniques of the engineer. Therefore, those skilled in the art can implement the present application set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the attached claims and their equivalents.

[0208] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0209] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0210] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0211] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0212] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0213] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0214] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A data search method, characterized in that: The method comprises: Obtain the administrator-defined lexical file and grammar file; wherein the lexical file is used to define the keywords and characters used in SQL, and the grammar file is used to define the syntax usage in SQL; Parsing the lexical file and the grammar file by using the Antrl tool to generate a corresponding grammar tree file; Generate a corresponding execution logic plan according to the syntax tree file, and associate the execution logic plan with Elasticsearch; Receive a user's search request for data in the Elasticsearch, respond to the search request according to the execution logic plan, and feed back search results.

2. A data search method according to claim 1, characterized in that: The lexical file and the grammar file are parsed by the Antrl tool to generate a corresponding grammar tree file, including: Generate a lexical analyzer and a syntax analyzer through the Antrl tool; Parsing the lexical file based on the lexical analyzer, converting the lexical rules in the lexical file into a token sequence; The tag sequence is input into the syntax analyzer, the syntax file is parsed based on the syntax analyzer, and the tag sequence is processed according to the syntax rules obtained by parsing to obtain a corresponding syntax tree file.

3. A data search method according to claim 1, characterized in that: The method further comprises: Obtaining identification information input by the administrator; the identification information is used to specify the Elasticsearch corresponding to the lexical file and the grammar file; The configuration of associating the execution logic plan with Elasticsearch includes: The execution logic plan is associated with the Elasticsearch corresponding to the identification information and configured.

4. A data search method according to claim 1, characterized in that: Before the step of receiving a user's search request for data in the Elasticsearch, the method further includes: Building a high-level API corresponding to the Elasticsearch; wherein the high-level API includes a RestHighLevelClient object; A connection is established between the high-level API and the Elasticsearch.

5. A data search method according to claim 4, characterized in that: The building of a high-level API corresponding to the Elasticsearch and establishing a connection with the Elasticsearch through the high-level API includes: Obtain the cluster information of the Elasticsearch; Create a RestClient object through the constructor mode, and pass the cluster information to the RestClient object; Create a RestHighLevelClient object, and pass the RestClient object to the RestHighLevelClient object; A connection is established between the RestHighLevelClient object and the Elasticsearch.

6. A data search method according to claim 5, characterized in that: The method further comprises: According to the cluster information, determine whether a user password is set in the Elasticsearch cluster; If a user password is set in the Elasticsearch cluster, create a BasicCredentialsProvider object, set the user password in the BasicCredentialsProvider object, and pass the BasicCredentialsProvider object to the RestClient object.

7. A data search method according to claims 1-6, characterized in that: The method further comprises: Create a database connection pool; According to the database connection pool, the connection of the Elasticsearch is optimized.

8. A data search system, characterized in that: The system comprises: An acquisition unit is used to acquire a lexical file and a grammar file customized by an administrator; wherein the lexical file is used to define keywords and characters used in SQL, and the grammar file is used to define the usage of grammar in SQL; A parsing unit, used to parse the lexical file and the grammar file through an Antrl tool to generate a corresponding grammar tree file; A processing unit, configured to generate a corresponding execution logic plan according to the syntax tree file, and associate the execution logic plan with Elasticsearch; The response unit is used to receive a user's search request for data in the Elasticsearch, respond to the search request according to the execution logic plan, and feed back search results.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a data search method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to implement a data search method as described in any one of claims 1 to 7 when executed by the processor.

Images