Framework technology implementation method for unifying query grammar and database query system

Abstract

The invention discloses a framework technology implementation method for unifying query grammar and a corresponding database query system. The JDBC parser parses the task information to obtain JDBC connection information and a task SQL (Structured Query Language); initializing database connection information according to the JDBC connection information, sending a task SQL to an SQL syntax tree analyzer for syntax tree analysis, and sending an SQL syntax tree generated by analysis and a database type to an SQL syntax tree converter; an execution statement is generated according to the SQL syntax tree, and the execution statement and a database type are sent to a task executor; connecting a corresponding database according to the database connection information; after receiving the execution statement, the task executor calls the database according to the type; the database returns an execution result to the task executor; the return result is sent to the corresponding result converter according to the database type; and the result is responded to the user. According to the method, the difference between the SQL grammar is shielded, and the defect that the component is not provided with the SQL is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of database systems, and in particular to a framework technology implementation method for unified query grammar and a database query system. Background Art

[0002] The goal of unified query syntax parsing technology is to achieve query capabilities for multiple data sources, including relational databases, NoSQL databases, big data computing engines, etc. Its development is based on the standardization of SQL or other query pre-research to achieve compatibility and interoperability of different data storage systems. Its ultimate goal is to achieve unified data processing and simplify the complexity of cross-data source queries.

[0003] With the development of query optimization technology, unified query syntax frameworks can not only parse queries, but also optimize them to improve query performance and reduce resource consumption. Some of these frameworks have begun to integrate machine learning models and artificial intelligence algorithms to provide more advanced data exploration and analysis capabilities. And with the popularization of cloud computing, unified query syntax frameworks are moving towards cloud native, supporting cloud services and containerized deployment in order to improve scalability and elasticity.

[0004] Although the unified query syntax framework technology has made some progress, it still faces some technical challenges, such as syntax and functional differences between different data sources, performance optimization, and support for emerging data types.

[0005] Traditional SQL queries rely on complete database protocols. For example, data is stored in relational or non-relational databases such as MySQL and Oracle. Although there is a standard SQL syntax, due to differences in SQL syntax between different relational databases and the fact that non-relational databases usually do not support SQL syntax without the introduction of third-party plug-ins, this leads to high data migration costs, high operation and maintenance costs, and high technical requirements for operation and maintenance personnel.

[0006] When processing massive amounts of data, relational or non-relational databases cannot meet actual business needs, so it is necessary to use the technical components of the big data ecosystem to solve actual business needs. This will lead to another problem, that is, when using the technical components of the big data ecosystem, some components come with SQL, such as Hive, Spark, Flink, etc., and some components do not come with SQL, such as Kafka and HBase.

[0007] Therefore, in this case, it is necessary to write different business logic interfaces to achieve data interaction between different components. If the complexity of business requirements increases with the increase of such components, the complexity of each set of interfaces will also increase, which will be extremely inconvenient for subsequent expansion and maintenance. Summary of the Invention

[0008] In response to this, the present invention proposes a framework technology implementation method for a unified query syntax and a database query system. By constructing a set of unified SQL languages, the present invention not only shields the differences existing between SQL grammars, but also solves the defect that components do not carry SQL, improves the task processing efficiency, and effectively reduces the system maintenance cost, achieving the effects of cost reduction, quality improvement, and efficiency increase.

[0009] The technical solution of the present invention is as follows:

[0010] A framework technology implementation method for a unified query syntax, comprising the following steps:

[0011] The JDBC parser parses the task information initiated by the user to obtain JDBC connection information and task SQL;

[0012] The JDBC parser initializes the database connection information according to the JDBC connection information and sends the task SQL to the SQL syntax tree parser;

[0013] The SQL syntax tree parser performs a syntax tree parsing on the task SQL, parses out the database type according to the task information, and then uniformly sends the parsed SQL syntax tree and the database type to the SQL syntax tree converter;

[0014] The SQL syntax tree converter generates corresponding execution statements according to the SQL syntax tree, sends the execution statements and the database type to the task executor; and connects to the corresponding database according to the database connection information;

[0015] After receiving the execution statements, the task executor will execute the task according to the execution statements by calling the corresponding database according to the database type;

[0016] The database returns the executed result to the task executor;

[0017] The task executor sends the returned result to the corresponding result converter according to the database type;

[0018] Finally, the result converter responds the result converted by it to the user.

[0019] In one embodiment, the JDBC parser initializes the database connection information according to the JDBC connection information; the specific method is:

[0020] Initialize the database connection information, where the database connection information includes the JDBC connection address, username, and password, and then send the database connection information to the database connection information factory.

[0021] In one embodiment, the SQL syntax tree converter generates corresponding execution statements according to the SQL syntax tree, sends the execution statements and the database type to the task executor; and connects to the corresponding database according to the database connection information; the specific method is as follows:

[0022] Send the SQL syntax tree to the corresponding SQL syntax tree converter according to the database type for conversion to generate corresponding execution statements, and send the execution statements and the database type to the task executor;

[0023] The SQL syntax tree converter receives the database connection information sent by the database connection information factory, and connects to the corresponding database according to the database connection information.

[0024] In one embodiment, the database includes a relational database and / or a non-relational database.

[0025] In one embodiment, the task information is initiated by the user through a third-party application.

[0026] In one embodiment, the result is responded to the user through a third-party application.

[0027] The present invention provides a database query system based on a unified query syntax framework, which includes:

[0028] A JDBC parser, whose function is: parse the task information initiated by the user to obtain JDBC connection information and task SQL; the JDBC parser initializes the database connection information according to the JDBC connection information, and sends the task SQL to the SQL syntax tree parser;

[0029] An SQL syntax tree parser, whose function is: perform syntax tree parsing on the task SQL, parse out the database type according to the task information, and then uniformly send the parsed SQL syntax tree and the database type to the SQL syntax tree converter;

[0030] An SQL syntax tree converter, whose function is: generate corresponding execution statements according to the SQL syntax tree, send the execution statements and the database type to the task executor; and connect to the corresponding database according to the database connection information;

[0031] A task executor, whose function is: after receiving the execution statement, call the corresponding database according to the database type;

[0032] The database returns the result after execution to the task executor;

[0033] The task executor sends the returned result to the corresponding result converter according to the database type;

[0034] Result converter, whose function is to respond the converted result to the user.

[0035] The present invention also provides a computer-readable storage medium, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, the above-mentioned method is implemented.

[0036] The present invention can be deployed in a big data operation platform and applied to fields such as enterprise digitization, rail transit, smart city, epidemic prevention and control, and anti-fraud. In the actual modeling process, through the framework technology of the unified query syntax implemented by the present invention, by uniformly converting different query syntaxes used by users, a set of unified query syntax is constructed to access different types of data sources, including relational databases, NoSQL databases, data warehouses, and big data platforms, without the need to learn multiple query languages. It reduces the dependence on a specific technology stack, lowers the data integration and migration costs, and also reduces the need for training and maintenance. While improving the task processing efficiency, it effectively reduces the system maintenance cost, achieving the effects of cost reduction, quality improvement, and efficiency increase. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is an exemplary system architecture diagram in which the present application can be applied;

[0038] Figure 2 is an architecture diagram of the implementation method of the framework technology of the unified query syntax of the present invention;

[0039] Figure 3 is an architecture diagram of the implementation system of the framework technology of the unified query syntax of an embodiment of the present application;

[0040] Figure 4 is a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The present application will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, rather than limiting the invention. Additionally, it should be noted that for the sake of description, only parts related to the relevant invention are shown in the drawings.

[0042] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and embodiments.

[0043] Figure 1 Shows an exemplary system architecture 100 for implementing the method of the framework technology of the unified query syntax to which the embodiments of the present application can be applied.

[0044] As Figure 1 shown, the system architecture 100 may include a data server 101, a network 102, and a main server 103. The network 102 is used to provide a medium for a communication link between the data server 101 and the main server 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0045] The main server 103 may be a server that provides various services, such as a data processing server that processes the information uploaded by the data server 101. The data processing server may implement a framework technology for a unified query syntax and store the processing results in a matching result library in an associated manner.

[0046] It should be noted that the method for implementing the framework technology of the unified query syntax provided by the embodiments of the present application is generally executed by the main server 103. Correspondingly, the system for implementing the framework technology of the unified query syntax is generally set in the main server 103.

[0047] It should be noted that the data server and the main server may be hardware or software. When it is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or it can be implemented as a single server. When it is software, it can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or it can be implemented as a single software or software module.

[0048] It should be understood that Figure 1 the numbers of the data server, the network, and the main server in are merely illustrative. According to the implementation requirements, there may be any number of terminal devices, networks, and servers.

[0049] The present invention provides a method for implementing a framework technology of a unified query syntax and a corresponding database query system.

[0050] According to Figure 1 the shown architecture diagram, the implementation method is specifically described as follows:

[0051] Step 1, the user initiates task information through a third-party application. The task information includes JDBC connection information and a specific task execution SQL;

[0052] Step 2, after receiving the task information and parsing the task information, the JDBC parser will perform the following two operations simultaneously:

[0053] 2.1 Initialize the database connection information, including the JDBC connection address, username, and password, and then send it to the database connection information factory;

[0054] 2.2 Send the database type parsed from the task information (abbreviated as "DBType", the same below) to the SQL syntax tree parser;

[0055] Then obtain the specific task execution SQL through the task information, and generate an SQL syntax tree by parsing.

[0056] Step 3, the SQL syntax tree parser parses the obtained task SQL for the syntax tree, and then sends the generated SQL syntax tree and DBType to the SQL syntax tree converter in a unified manner;

[0057] Step 4, the SQL syntax tree converter will perform the following two operations:

[0058] 4.1 Send the received SQL syntax tree to the corresponding SQL syntax tree converter for conversion according to DBType, and send the executed statement and DBType after successful conversion to the task executor;

[0059] 4.2 Receive the connection information sent by the database connection information factory, and connect to the corresponding relational database or non-relational database according to the connection information.

[0060] Step 5, after receiving the execution statement, the task executor will call the corresponding relational database or non-relational database according to DBType;

[0061] Step 6, the relational database or non-relational database returns the executed result to the task executor;

[0062] Step 7, the task executor sends the returned result to the corresponding result converter according to DBType;

[0063] Step 8, the final result converter responds the converted result to the user through a third-party application.

[0064] By executing the above steps, a series of problems brought by the differences in SQL syntax between different relational databases and the non-support for SQL syntax by non-relational databases without introducing third-party plugins can be shielded.

[0065] The framework technology implementation method and corresponding system for unified query syntax proposed by the present invention construct a unified query syntax framework through a series of steps, shield the SQL syntax differences between different databases, solve the problem that components do not carry SQL, and achieve remarkable technical effects in multiple fields, specifically including the following:

[0066] 1. Multi-data source access and syntax unification: It can access different types of data sources such as relational databases, NoSQL databases, data warehouses, and big data platforms, and constructs a unified query syntax to achieve the unification of query capabilities for multiple data sources. Users do not need to learn multiple query languages, reducing the dependence on specific technology stacks, lowering the costs of data integration and migration, and also reducing the need for training and maintenance.

[0067] 2. Efficient task processing and performance improvement: It improves the task processing efficiency. Through a unified process, it processes the task information initiated by users. From parsing the task information to executing the query and returning the results, each component works together to optimize the task execution process. It supports query optimization to improve query performance and reduce resource consumption. Some frameworks also integrate machine learning models and artificial intelligence algorithms to provide more advanced data exploration and analysis capabilities.

[0068] 3. Adapt to multiple technology development trends: With the popularization of cloud computing, the framework develops towards cloud-native, supports cloud services and containerized deployment, improves scalability and elasticity, and can better meet the development needs of modern information technology architectures. It can be deployed in big data operation platforms and has been verified in actual combat modeling work in fields such as enterprise digitization, rail transit, smart cities, epidemic prevention and control, and anti-fraud, providing an efficient and unified solution for data processing in these fields and helping related work to be carried out efficiently.

[0069] 4. Cost reduction and improved maintainability: It effectively reduces the system maintenance cost, avoiding the writing of complex business logic interfaces caused by syntax differences in different data sources and components without SQL, as well as the resulting high maintenance cost problems. As the number of components increases and the complexity of business requirements rises, this solution can avoid the increase in the complexity of each set of interfaces, facilitating subsequent expansion and maintenance, and ensuring the stability and efficiency of the system during long-term operation and business development.

[0070] Correspondingly, Figure 3 is the system architecture diagram of the framework technology implementation of the unified query syntax of an embodiment of the present application. As Figure 3 shown, the present invention also proposes a database query system based on a unified query syntax framework, which includes:

[0071] A JDBC parser 301, whose functions are: parsing the task information initiated by users to obtain JDBC connection information and task SQL; the JDBC parser initializes the database connection information according to the JDBC connection information and sends the task SQL to the SQL syntax tree parser.

[0072] SQL syntax tree parser 302, whose function is: to perform syntax tree parsing on the task SQL, parse out the database type according to the task information, and then uniformly send the generated SQL syntax tree and database type to the SQL syntax tree converter.

[0073] SQL syntax tree converter 303, whose function is: to generate corresponding execution statements according to the SQL syntax tree, send the execution statements and database type to the task executor; and connect to the corresponding database according to the database connection information.

[0074] Task executor 304, whose function is: after receiving the execution statement, it will call the corresponding database according to the database type.

[0075] The database returns the result after execution to the task executor.

[0076] The task executor sends the returned result to the corresponding result converter according to the database type.

[0077] Result converter, whose function is: to respond the converted result to the user.

[0078] The following refers to Figure 4 , which shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. Figure 4 The shown electronic device is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0079] As Figure 4 shown, the computer system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage section 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the system 400 are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other through a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0080] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including, for example, a liquid crystal display (LCD) and a speaker; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0081] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable storage medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the methods of the present application are performed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable storage medium other than the computer-readable storage medium, and the computer-readable storage medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0082] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0083] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0084] The modules described in the embodiments of this application can be implemented in software or in hardware.

[0085] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device is enabled to: the JDBC parser parses the task information to obtain JDBC connection information and task SQL; initializes the database connection information according to the JDBC connection information, sends the task SQL to the SQL syntax tree parser for syntax tree parsing, and sends the parsed SQL syntax tree and database type to the SQL syntax tree converter; it generates an execution statement according to the SQL syntax tree, sends the execution statement and database type to the task executor; and connects to the corresponding database according to the database connection information; after receiving the execution statement, the task executor calls the database according to the type; the database returns the execution result to the task executor; it sends the return result to the corresponding result converter according to the database type; and responds the result to the user.

[0086] The above description is only the preferred embodiment of the present application and the explanation of the applied technical principle. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present application.

Claims

1. A framework technology implementation method for unified query grammar, characterized in that: It includes the following steps: The JDBC parser parses the task information initiated by the user to obtain the JDBC connection information and task SQL; The JDBC parser initializes the database connection information according to the JDBC connection information and sends the task SQL to the SQL syntax tree parser; The SQL syntax tree parser performs syntax tree parsing on the task SQL, parses out the database type according to the task information, and then sends the SQL syntax tree and database type generated by the parsing to the SQL syntax tree converter; The SQL syntax tree converter generates the corresponding execution statement according to the SQL syntax tree, sends the execution statement and database type to the task executor, and connects to the corresponding database according to the database connection information; After receiving the execution statement, the task executor will call the corresponding database according to the database type to execute the task according to the execution statement; The database returns the execution results to the task executor; The task executor sends the returned results to the corresponding result converter according to the database type; Finally, the result converter responds to the user with the converted result.

2. The method according to claim 1, characterized in that The JDBC parser initializes the database connection information according to the JDBC connection information; the specific method is: Initialize database connection information, the database connection information includes JDBC connection address, user name and password, and then send the database connection information to the database connection information factory.

3. The method according to claim 2, characterized in that The SQL syntax tree converter generates a corresponding execution statement according to the SQL syntax tree, sends the execution statement and the database type to the task executor, and connects to the corresponding database according to the database connection information; the specific method is: Sending the SQL syntax tree to a corresponding SQL syntax tree converter according to the database type for conversion to generate a corresponding execution statement, and sending the execution statement and the database type to a task executor; The SQL syntax tree converter receives the database connection information sent by the database connection information factory, and connects to the corresponding database according to the database connection information.

4. The method according to any one of claims 1 to 3, characterized in that: The database includes a relational database and / or a non-relational database.

5. The method according to any one of claims 1 to 3, characterized in that: The task information is initiated by the user through a third-party application.

6. The method according to any one of claims 1 to 3, characterized in that: The result is responded to the user through the third-party application.

7. A database query system based on a unified query grammar framework, characterized in that: It includes: JDBC parser, whose functions are: parsing the task information initiated by the user to obtain JDBC connection information and task SQL; The JDBC parser initializes the database connection information according to the JDBC connection information and sends the task SQL to the SQL syntax tree parser; The SQL syntax tree parser has the following functions: parsing the task SQL into a syntax tree, parsing the database type according to the task information, and then sending the SQL syntax tree and database type generated by the parsing to the SQL syntax tree converter; The SQL syntax tree converter generates the corresponding execution statement according to the SQL syntax tree, sends the execution statement and database type to the task executor, and connects to the corresponding database according to the database connection information. The task executor has the following functions: after receiving the execution statement, it will call the corresponding database according to the database type; The database returns the execution results to the task executor; The task executor sends the returned results to the corresponding result converter according to the database type; The result converter has the function of responding the conversion result to the user.

8. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 6 is implemented.

Images