HTTP query service interface arrangement method and system based on SQL grammar
Through the HTTP query service interface orchestration method based on SQL syntax, the problem that HTTP service interface in the prior art is difficult to meet diverse needs, and the automated orchestration of the interface and the flexibility of the system are improved.
Patent Information
- Application Number
- CN202411922970.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-25
- Publication Date
- 2025-05-09
AI Technical Summary
The prior art is difficult to meet the diverse HTTP service interface needs of third-party systems or users, resulting in increased system integration complexity and development costs.
Using the HTTP query service interface orchestration method based on SQL syntax, the HTTP query service interface information is entered, abstracted into a virtual data table structure, and the query plan is generated using the Calcite framework, and the HTTP request is executed according to the query plan, and the integrated result is returned to the user.
It realizes the automation and standardized orchestration of interfaces, simplifies the complexity of interface integration, and improves the flexibility and scalability of the system.
Smart Images
Figure CN119961333A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of big data, and in particular to a method and system for arranging an HTTP query service interface based on SQL grammar. Background Art
[0002] With the deepening of informatization construction in traditional industries and the development of big data technology, the service platform has become one of the key infrastructures for the digital transformation of enterprises. The service platform aims to publish data resources for big data governance and data analysis modeling in the form of HTTP service interfaces, and proxy third-party REST interfaces to achieve open sharing of data. However, in actual applications, due to the different data interface standards and data specifications between systems, the directly published HTTP service interface is often difficult to meet the diverse needs of third-party systems or users. Users often need to perform secondary processing on query results, or need to associate multiple HTTP service interfaces, or even use the query results of one interface as parameters to query other interfaces. These requirements increase the complexity of system integration and development costs.
[0003] At present, there are some interface orchestration tools on the market, such as YAPI, Apifox, etc. These tools usually provide interface management as their main function, and perform interface orchestration through graphical interfaces or scripting languages. However, they generally lack a unified and easy-to-understand orchestration language, and it is difficult to achieve flexible expression of complex business logic. SQL, as a widely used query language, has the advantages of simple syntax, easy to understand, and powerful functions. By introducing it into the field of HTTP service interface orchestration, the complexity and learning cost of interface orchestration can be greatly reduced.
[0004] With the continuous development of big data and cloud computing technologies, the number and types of service interfaces will continue to grow, and the demand for interface orchestration will become more urgent. Summary of the invention
[0005] In order to solve the technical problems raised by the background technology, the present invention proposes a method and system for arranging HTTP query service interfaces based on SQL syntax to solve the above technical problems.
[0006] According to one aspect of the present invention, a method for arranging an HTTP query service interface based on SQL syntax is proposed, comprising:
[0007] S1. Enter some HTTP query service interface information, which includes interface input parameters and interface output parameters;
[0008] S2. Abstracting several pieces of HTTP query service interface information into corresponding virtual data table structures respectively, wherein the virtual data table structure uses the interface input parameters and interface output parameters of the corresponding HTTP query service interface information as table fields;
[0009] S3. Generate a query plan using the Calcite framework based on the SQL statement and virtual data table structure input by the user. The query plan includes the HTTP query service interface sequence to be called, interface call parameters, and data processing logic.
[0010] S4, traverse each node in the abstract syntax tree after metadata verification, perform relational algebra transformation on the corresponding node, and obtain a relational algebra expression tree;
[0011] S5, traverse the relational algebra expression tree, execute the HTTP request in the order of nodes, and obtain the HTTP request data;
[0012] S6. Integrate the HTTP request data according to the preset requirements of the SQL statement, and return the integrated results to the user.
[0013] The calling logic of HTTP service interface is defined and arranged through SQL syntax, realizing the automatic and standardized arrangement of interface. This method and system have the advantages of being easy to use, powerful, flexible and scalable, which can greatly simplify the complexity of interface integration and improve the flexibility and scalability of the system.
[0014] In some specific embodiments, step S1 HTTP query service interface information includes interface name, interface address and return data type.
[0015] In some specific embodiments, step S3 includes:
[0016] S31. Parse the SQL statement through the SQL parser of the Calcite framework and convert the SQL statement into an abstract syntax tree. Each node in the abstract syntax tree is an SQL node. The SQL query keywords include SELECT node, FROM node, JOIN node, ON node and ORDER BY node.
[0017] S32, verifying metadata of the abstract syntax tree, where the metadata includes table name, field name, function name, and data type;
[0018] S33. Register by presetting the class name, method name and parameter type of the UDF function in the abstract syntax tree.
[0019] The Java Calcite framework is used for SQL syntax parsing and verification to ensure the correctness and compatibility of SQL statements. Calcite is an open source SQL parser and query optimizer framework that supports multiple data sources and query engines and can meet the requirements of the present invention for SQL syntax processing.
[0020] In some specific embodiments, step S33 includes dynamic registration and static registration; wherein, dynamic registration includes passing UDF information through the execution plan during execution, and static registration includes adding UDF when the Schema is initialized. Dynamic registration allows UDF information to be passed through the execution plan during query execution, which allows flexible addition or modification of UDFs at runtime. This flexibility is very useful for scenarios that require frequent changes or testing of different UDFs. Statically registered UDFs are added to the system when the Schema is initialized, so these functions can be called directly during query execution without additional parsing and lookup operations. This helps to improve query execution efficiency and response speed.
[0021] In some specific embodiments, step S4 includes: generating corresponding projection nodes for SELECT nodes; generating corresponding scan and join nodes for FROM nodes and JOIN nodes; and converting ON nodes into filter nodes or as part of JOIN nodes.
[0022] In some specific embodiments, step S5 of executing HTTP requests in node order includes: pre-acquiring interfaces that need to be called in subsequent queries according to the query plan, and executing the interfaces that need to be called in subsequent queries in advance, thereby reducing waiting time.
[0023] In some specific embodiments, step S6 includes preprocessing the HTTP request data, and the preprocessing includes filtering, sorting, and dictionary translation.
[0024] According to a second aspect of the present invention, a SQL-grammar-based HTTP query service interface orchestration system is proposed, the system comprising:
[0025] The HTTP interface input module is configured to input a number of HTTP query service interface information, where the HTTP query service interface information includes interface input parameters and interface output parameters;
[0026] The HTTP interface abstraction module is configured to abstract a plurality of HTTP query service interface information into corresponding virtual data table structures, wherein the virtual data table structure uses the interface input parameter and the interface output parameter of the corresponding HTTP query service interface information as table fields;
[0027] A query plan generation module is configured to generate a query plan using the Calcite framework according to the SQL statement and virtual data table structure input by the user. The query plan includes a sequence of HTTP query service interfaces to be called, interface call parameters, and data processing logic.
[0028] A relational algebra conversion module is configured to traverse each node in the abstract syntax tree after metadata verification, perform relational algebra conversion on the corresponding node, and obtain a relational algebra expression tree;
[0029] An execution module is configured to traverse the relational algebra expression tree, execute the HTTP request in node order, and obtain HTTP request data;
[0030] The output module is configured to integrate the HTTP request data according to the preset requirements of the SQL statement and return the integrated results to the user.
[0031] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, any of the above methods is implemented.
[0032] According to a fourth aspect of the present invention, an electronic device is proposed, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.
[0033] The method and system for programming HTTP query service interfaces based on SQL syntax in this application achieve the following beneficial effects:
[0034] The calling logic of HTTP service interface is defined and arranged through SQL syntax, realizing the automatic and standardized arrangement of interface. This method and system have the advantages of being easy to use, powerful, flexible and scalable, which can greatly simplify the complexity of interface integration and improve the flexibility and scalability of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description are used to explain the principles of the present invention. Other embodiments and many expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:
[0036] Figure 1It is a flowchart of a method for arranging an HTTP query service interface based on SQL syntax according to an embodiment of the present application;
[0037] Figure 2 It is a schematic diagram of the SQL syntax parsing and HTTP interface request processing flow of an embodiment of the present application;
[0038] Figure 3 This is a schematic diagram of the first HTTP query service interface information of an embodiment of the present application;
[0039] Figure 4 This is a schematic diagram of the second HTTP query service interface information of an embodiment of the present application;
[0040] Figure 5 It is a reference diagram of the meaning of a relational algebra expression tree of an embodiment of the present application;
[0041] Figure 6 This is a schematic diagram of an SQL statement of an embodiment of the present application;
[0042] Figure 7 is a relational algebra expression tree diagram of an embodiment of the present application;
[0043] Figure 8 It is a schematic diagram of an HTTP query service interface arrangement system based on SQL syntax according to an embodiment of the present application;
[0044] Fig. 9 This is an architecture diagram of an HTTP query service interface orchestration system based on SQL syntax according to an embodiment of the present application;
[0045] Fig.10 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION
[0046] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0047] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0048] According to an embodiment of the present application, a method for arranging an HTTP query service interface based on SQL grammar is provided. Figure 1is a flowchart of a method for arranging an HTTP query service interface based on SQL syntax according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the SQL syntax parsing and HTTP interface request processing flow of an embodiment of the present application. Figure 1 and Figure 2 As shown, the method includes:
[0049] S1. Enter some HTTP query service interface information, which includes interface input parameters and interface output parameters;
[0050] Specifically, the HTTP query service interface information in step S1 includes the interface name, interface address and return data type. These interface information will be used for subsequent SQL statement parsing and data table abstraction.
[0051] As an example, HTTP query service interface information such as Figure 3 and Figure 4 As shown in the figure, they all include interface name, interface key, interface address, interface method, interface input parameter and interface output parameter.
[0052] S2. Abstracting several pieces of HTTP query service interface information into corresponding virtual data table structures respectively, wherein the virtual data table structure uses the interface input parameters and interface output parameters of the corresponding HTTP query service interface information as table fields;
[0053] Each interface corresponds to a virtual table, and the request parameters and return fields of the interface correspond to the columns of the virtual table respectively.
[0054] As an example, by abstracting the interface key as the name of the virtual table, prefixing the input parameter with "p_", prefixing the return parameter with "r_", and using the input and return parameters as the fields of the table, we get the following two virtual tables:
[0055] Table idcard_person_info:
[0056]
[0057]
[0058] Table idcard_airline_booking_info:
[0059]
[0060] S3. Generate a query plan using the Calcite framework based on the SQL statement and virtual data table structure input by the user. The query plan includes the HTTP query service interface sequence to be called, interface call parameters, and data processing logic.
[0061] Specifically, step S3 includes:
[0062] SQL parsing: S31. Parse SQL statements through the SQL parser of the Calcite framework and convert them into an abstract syntax tree (AST). Each node in the abstract syntax tree (AST) is an SQL node (SqlNode). SQL query keywords include SELECT node, FROM node, JOIN node, ON node, and ORDER BY node.
[0063] It should be noted that Calcite is an open source SQL parser and query optimizer framework that supports multiple data sources and query engines and can meet the requirements of the present invention for SQL syntax processing.
[0064] Among them, the Calcite framework supports two SQL parsing methods based on Java CC and ANTLR.
[0065] Metadata verification: S32, verify the metadata of the abstract syntax tree (AST), which includes table name, field name, function name and data type;
[0066] UDF registration: S33. Register by presetting the class name, method name and parameter type of the UDF function in the abstract syntax tree (AST).
[0067] S4. Traverse each node in the abstract syntax tree after metadata verification, perform relational algebra conversion on the corresponding node, and obtain a relational algebra expression tree; the AST after metadata verification will be converted into a relational algebra tree, that is, a relational algebra expression tree composed of RelNode nodes. The Calcit framework uses SqlToRelConverter to convert SqlNode to RelNode. This process traverses the AST nodes based on the Visitor mode and generates corresponding RelNode nodes.
[0068] Specifically, step S4 includes: generating corresponding projection nodes for SELECT nodes; generating corresponding scan and join nodes for FROM nodes and JOIN nodes; and converting ON nodes into filter nodes or serving as parts of JOIN nodes.
[0069] Specifically, the meaning of the relational algebra expression tree is referenced Figure 5 , including various nodes for different operations on data.
[0070] Moreover, each node in the relational algebra expression tree represents a query operation, such as interface call, data filtering, result merging, etc.
[0071] S5, traverse the relational algebra expression tree, execute the HTTP request in the order of nodes, and obtain the HTTP request data;
[0072] Specifically, step S5 of executing HTTP requests in order of nodes includes: pre-acquiring interfaces that need to be called in subsequent queries according to the query plan, and executing the interfaces that need to be called in subsequent queries in advance, thereby reducing waiting time.
[0073] In addition, this application also proposes optimizations for the following issues:
[0074] 1. The query plan contains a large number of HTTP requests, and there is no effective caching or concurrent processing between these requests, which may cause slow system response; therefore, this application includes a caching mechanism: for frequently called interfaces and data, a caching mechanism is introduced to reduce unnecessary HTTP requests and improve response speed.
[0075] 2. Data processing complexity: For complex SQL queries, a large number of data processing operations may be required, such as filtering, sorting, and aggregation, which may increase the computing burden of the system; therefore, this application optimizes the data processing logic, such as using more efficient data structures and algorithms. For complex data processing operations, consider using distributed computing or parallel processing technology.
[0076] 3. Resource competition and blocking: If multiple queries are executed at the same time and they need to access the same resources (such as network, CPU or memory), resource competition and blocking may occur, which will affect system performance. Therefore, for scenarios where multiple HTTP requests need to be executed concurrently, this application uses a thread pool or asynchronous programming model to improve the parallel processing capability of queries.
[0077] S6. Integrate the HTTP request data according to the preset requirements of the SQL statement, and return the integrated results to the user.
[0078] Specifically, step S6 includes preprocessing the HTTP request data, and the preprocessing includes filtering, sorting and dictionary translation.
[0079] Through the above steps, the SQL syntax parsing processing module can realize the association, data processing and result integration of multiple HTTP query service interfaces, thereby improving the flexibility and efficiency of data query.
[0080] As an example, when the user needs to query the detailed information of the person corresponding to the given ID number and the flight information departing in January 2024, the following is obtained according to the demand: Figure 6 The SQL statement shown;
[0081] (1) In step S31, the SQL parser of Calcite parses the above SQL statement into an abstract syntax tree (AST), and each node in the abstract syntax tree (AST) is a SqlNode, such as SELECT, FROM, LEFT JOIN, ON, ORDER BY, etc.;
[0082] (2) Through the metadata verification in step S32, the Calcite framework is used to check whether the table names idcard_person_info and idcard_airline_booking_info exist; check whether the field names p_idCard, r_idCard, r_name, r_birth, r_sexCode, r_nationCode, r_eduBackgroundCode, r_address and r_idCard, r_airlineNo, r_startTime, r_arriveTime, r_seatNo, r_startStation, r_arriveStation, r_airlineCompany exist in the corresponding tables; check whether the UDF functions translateSex(), translateNation(), and translateEduBackground() exist; and verify whether the field types in the JOIN condition ON t1.`r_idCard`=t2.`r_idCard` match.
[0083] (3) Through step S4, the SqlToRelConverter method of the Calcite framework is used to traverse the nodes of the abstract syntax tree (AST). For the SELECT node, the corresponding Project RelNode is generated; for the FROM and JOIN nodes, the corresponding Scan and Join RelNode are generated; the ON condition t1.`r_idCard`=t2.`r_idCard` is converted to a Filter RelNode or as part of the Join condition. For translateSex(`r_sexCode`), Calcite generates a ScalarFunctionRelNode, which encapsulates the execution logic of translateSex; the conditions `r_startTime`>='2024-01-01 00:00:00'AND`r_startTime`<'2024-02-01 00:00:00' are represented by the FilterRel node; the SortRel node indicates that the filtered results are sorted in ascending order by the `r_startTime` field. Finally, the following is obtained: Figure 7 The relational algebra expression tree shown is:
[0084] According to a second aspect of the present invention, Figure 8 is a schematic diagram of an HTTP query service interface arrangement system based on SQL syntax according to an embodiment of the present application. Figure 8 As shown, the system includes:
[0085] HTTP interface input module 801 is configured to input a number of HTTP query service interface information, the HTTP query service interface information includes interface input parameters and interface output parameters;
[0086] The HTTP interface abstraction module 802 is configured to abstract a plurality of HTTP query service interface information into corresponding virtual data table structures, wherein the virtual data table structures use the interface input parameters and interface output parameters of the corresponding HTTP query service interface information as table fields;
[0087] The query plan generation module 803 is configured to generate a query plan using the Calcite framework according to the SQL statement and virtual data table structure input by the user, wherein the query plan includes a sequence of HTTP query service interfaces to be called, interface call parameters, and data processing logic;
[0088] The relational algebra conversion module 804 is configured to traverse each node in the abstract syntax tree after metadata verification, perform relational algebra conversion on the corresponding node, and obtain a relational algebra expression tree;
[0089] An execution module 805 is configured to traverse the relational algebra expression tree, execute the HTTP request in node order, and obtain HTTP request data;
[0090] The output module 806 is configured to integrate the HTTP request data according to the preset requirements of the SQL statement and return the integrated results to the user.
[0091] Optionally, the architecture diagram of the SQL-grammar-based HTTP query service interface orchestration system of this application consists of the following modules, refer to Fig. 9 ,include:
[0092] HTTP query service interface management module: used to store HTTP service interface information, including interface path, input and output parameter names, request method, request header information, etc. This module provides add, delete, modify and query interfaces to facilitate the management of interface information.
[0093] HTTP request client module: responsible for sending HTTP requests and receiving responses. This module supports multiple HTTP request methods (such as GET, POST, PUT, DELETE, etc.), can process HTTP request headers, request bodies and other information, and has exception handling mechanisms such as retries and timeouts.
[0094] SQL syntax parsing processing module: The core module is responsible for abstracting the data of the HTTP service interface storage module into a data table structure, generating corresponding HTTP requests by parsing the SQL statements entered by the user, and processing the returned results. This module uses the Java Calcite framework to perform SQL syntax parsing and syntax verification to ensure the correctness and legality of SQL statements.
[0095] The method and system for programming HTTP query service interfaces based on SQL syntax in this application achieve the following beneficial effects:
[0096] The calling logic of HTTP service interface is defined and arranged through SQL syntax, realizing the automatic and standardized arrangement of interface. This method and system have the advantages of being easy to use, powerful, flexible and scalable, which can greatly simplify the complexity of interface integration and improve the flexibility and scalability of the system.
[0097] Reference below Fig.10 , which shows a schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application. Fig.10 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0098] like Fig.10 As shown, the computer system includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the system 1000 are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0099] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.
[0100] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.
[0101] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0103] The modules involved in the embodiments of the present application may be implemented by software or by hardware.
[0104] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device executes: inputting several HTTP query service interface information, the HTTP query service interface information includes interface input parameters and interface output parameters; abstracting several HTTP query service interface information into corresponding virtual data table structures, and the virtual data table structure uses the interface input parameters and interface output parameters of the corresponding HTTP query service interface information as the fields of the table; according to the SQL statement and the virtual data table structure input by the user, using the Calcite framework to generate a query plan, the query plan includes the HTTP query service interface sequence to be called, the interface call parameters and the data processing logic; traversing each node in the abstract syntax tree after metadata verification, performing relational algebra transformation on the corresponding node, and obtaining a relational algebra expression tree; traversing the relational algebra expression tree, executing HTTP requests in node order, and obtaining HTTP request data; integrating the HTTP request data according to the preset requirements of the SQL statement, and returning the integrated results to the user.
[0105] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.
Claims
1. A method for arranging HTTP query service interface based on SQL grammar, characterized in that: include: S1. Enter some HTTP query service interface information, where the HTTP query service interface information includes interface input parameters and interface output parameters; S2. Abstracting several pieces of HTTP query service interface information into corresponding virtual data table structures respectively, wherein the virtual data table structure uses the interface input parameter and interface output parameter of the corresponding HTTP query service interface information as table fields; S3. Generate a query plan using the Calcite framework according to the SQL statement input by the user and the virtual data table structure. The query plan includes a sequence of HTTP query service interfaces to be called, interface call parameters, and data processing logic. S4, traverse each node in the abstract syntax tree after metadata verification, perform relational algebra transformation on the corresponding node, and obtain a relational algebra expression tree; S5, traversing the relational algebra expression tree, executing HTTP request in node order, and obtaining HTTP request data; S6. Integrate the HTTP request data according to the preset requirements of the SQL statement, and return the integrated results to the user.
2. The method according to claim 1, characterized in that The HTTP query service interface information in step S1 includes the interface name, interface address and return data type.
3. The method according to claim 1, characterized in that Step S3 includes: S31, parsing the SQL statement by using the SQL parser of the Calcite framework, converting the SQL statement into an abstract syntax tree, wherein each node in the abstract syntax tree is an SQL node, and the SQL query keywords include a SELECT node, a FROM node, a JOIN node, an ON node, and an ORDER BY node; S32, performing metadata verification on the abstract syntax tree, where the metadata includes table name, field name, function name and data type; S33, registering by presetting the class name, method name and parameter type of the UDF function in the abstract syntax tree.
4. The method according to claim 3, characterized in that Step S33 includes dynamic registration and static registration; wherein the dynamic registration includes passing UDF information through the execution plan during execution, and the static registration includes adding UDF when the Schema is initialized.
5. The method according to claim 1, characterized in that The relational algebra transformation in step S4 includes: generating corresponding projection nodes for the SELECT nodes; generating corresponding scan and join nodes for the FROM nodes and the JOIN nodes; and converting the ON nodes into filter nodes or as part of the JOIN nodes.
6. The method according to claim 1, characterized in that The step S5 of executing the HTTP request in the order of nodes includes: pre-acquiring the interface that needs to be called in the subsequent query according to the query plan, and executing the interface that needs to be called in the subsequent query in advance.
7. The method according to claim 1, characterized in that Step S6 includes preprocessing the HTTP request data, wherein the preprocessing includes filtering, sorting and dictionary translation.
8. An HTTP query service interface arrangement system based on SQL syntax, characterized in that: The system comprises: An HTTP interface input module is configured to input a number of HTTP query service interface information, wherein the HTTP query service interface information includes interface input parameters and interface output parameters; The HTTP interface abstraction module is configured to abstract a plurality of HTTP query service interface information into corresponding virtual data table structures, wherein the virtual data table structures use the interface input parameters and interface output parameters of the corresponding HTTP query service interface information as table fields; A query plan generation module is configured to generate a query plan using the Calcite framework according to the SQL statement input by the user and the virtual data table structure, wherein the query plan includes a sequence of HTTP query service interfaces to be called, interface call parameters, and data processing logic; A relational algebra conversion module is configured to traverse each node in the abstract syntax tree after metadata verification, perform relational algebra conversion on the corresponding node, and obtain a relational algebra expression tree; An execution module is configured to traverse the relational algebra expression tree, execute HTTP requests in node order, and obtain HTTP request data; The output module is configured to integrate the HTTP request data according to the preset requirements of the SQL statement and return the integrated results to the user.
9. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 7 is implemented.
10. An electronic device, comprising: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.