Data query method and apparatus

Abstract

The application discloses a data query method and device, which can be applied to the financial field or other technical fields, and comprises the following steps: sending a query request body of a user to a corresponding query engine, so that the query engine determines structured data of a service corresponding to the query request body; generating an SQL statement of the service according to the structured data; and performing data query according to the SQL statement to obtain a data query result. The application helps to improve the arrangement and publishing efficiency of the service.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data query, in particular to a data query method and device. BACKGROUND

[0002] The existing service orchestration platform forms a service through a visual interface by user drag and drop, and then publishes the service, and the service orchestration is formed through a given fixed SQL. The current service orchestration mode is too fixed and not flexible, and the fixed SQL corresponding to the service needs to be fixed each time the service is orchestrated, which requires the service orchestration personnel to manually write the SQL statement, and if the service query condition changes, the corresponding SQL also needs to be changed. As can be seen, the existing service orchestration and publishing mode depends on manual writing of SQL statements, which is time-consuming and laborious and not flexible. SUMMARY

[0003] The present application is proposed to solve at least one of the technical problems in the background art.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a data query method is provided, which comprises:

[0005] sending a query request body of a user to a corresponding query engine to enable the query engine to determine the structured data of the service corresponding to the query request body;

[0006] generating an SQL statement of the service according to the structured data;

[0007] performing data query according to the SQL statement to obtain a data query result.

[0008] Optionally, the data query method further comprises:

[0009] obtaining orchestration information of a service to be created, wherein the orchestration information is obtained by detecting the operation of the user on the service orchestration interface;

[0010] generating the structured data of the service to be created according to the orchestration information to complete the creation of the service;

[0011] establishing a corresponding relationship between the completed service and the query engine according to the field of the completed service.

[0012] Optionally, after the creation of the service is completed, it further comprises:

[0013] The service call document of the created service is generated to enable a user to make a service query according to the service call document, wherein the service call document comprises a request address of the service, a query request body of the service, a return structure, optional parameters and mandatory parameters.

[0014] Optionally, the SQL statement of the service is generated according to the structured data, and specifically comprises:

[0015] The structured data is parsed according to Druid SQL and an SQL statement is constructed.

[0016] Optionally, the SQL statement is constructed, and specifically comprises: constructing a field list, constructing a field list prefix, constructing a query source list, constructing a correlation condition, constructing a where condition, constructing a grouping field and condition, and constructing a sorting field.

[0017] Optionally, the structured data comprises a service ID, a referenced service ID, a query type, a field list of a query, a table to be queried, a query condition, and a sorting condition.

[0018] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a data query device is provided, which comprises:

[0019] A structured data determination module is configured to send a query request body of a user to a corresponding query engine to enable the query engine to determine structured data of a service corresponding to the query request body;

[0020] An SQL statement generation module is configured to generate an SQL statement of the service according to the structured data;

[0021] A data query module is configured to make a data query according to the SQL statement to obtain a data query result.

[0022] Optionally, the data query device further comprises:

[0023] An arrangement information acquisition module is configured to acquire arrangement information of a service to be created, wherein the arrangement information is obtained by detecting an operation of a user on a service arrangement interface;

[0024] A service creation module is configured to generate structured data of the service to be created according to the arrangement information to complete creation of the service;

[0025] A query engine association module is configured to establish a corresponding relationship between the created service and a query engine according to a field of the created service.

[0026] In order to achieve the above object, according to another aspect of the present application, there is further provided a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above data query method when executing the computer program.

[0027] In order to achieve the above object, according to another aspect of the present application, there is further provided a computer readable storage medium storing a computer program, wherein the computer program implements the steps in the above data query method when executed in a computer processor.

[0028] The present application has the following beneficial effects:

[0029] In the present application, when performing service query, a query engine is used to determine the structured data of the service corresponding to the query request of the user, and then generate the SQL statement of the service according to the structured data, and finally perform data query according to the SQL statement to obtain the data query result. The present application sets a data structure, and then converts the service into structured data, so that the SQL statement can be directly generated according to the structured data when performing service query, which solves the defect of manual writing of SQL statement in the prior art, and helps to improve the efficiency of service arrangement and release. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor. In the drawings:

[0031] Figure 1 is the first flow chart of the data query method of the present application embodiment;

[0032] Figure 2 is the second flow chart of the data query method of the present application embodiment;

[0033] Figure 3 is the first structural block diagram of the data query device of the present application embodiment;

[0034] Figure 4 is the second structural block diagram of the data query device of the present application embodiment;

[0035] Figure 5 is the schematic diagram of the computer device of the present application embodiment. DETAILED DESCRIPTION

[0036] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work should belong to the protection scope of the present application.

[0037] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0038] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification and claims of the present application and the above-described drawings are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not have to be limited to the clearly listed steps or units, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0039] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0040] It should be noted that the data query method and device of the present application can be used in the financial field, and can also be applied to other technical fields.

[0041] The present application aims at the shortcomings of existing service arrangement, and the data structure of the service is not abstracted. In order to solve the development of repeated code, the process of service arrangement publishing is abstracted into a unified data structure, SQL is constructed and parsed through a unified engine, and a general query request body (query Schema) is dynamically constructed. Flexible data structure can quickly publish to form its own service, and the query Schema can be flexibly defined according to the user's own demand. The existing platform still needs to develop the specific logic of the service for the general query, and the query of the service is not flexible enough, and does not have the characteristics of flexible query. If the query condition changes, the related logic code needs to be adjusted again. Therefore, the present application designs a general service arrangement data structure, provides a method for quickly publishing services, flexible query execution of services, and flexible configuration of service authentication and flow limiting, improves the convenience of application development, and effectively reduces the maintenance and management cost of repeated code.

[0042] Figure 1 is the first flowchart of the data query method of the embodiment of the present application, as shown in Figure 1 in an embodiment of the present application, the data query method of the present application comprises steps S101 to S103.

[0043] Step S101, the query request body of the user is sent to the corresponding query engine, so that the query engine determines the structured data of the service corresponding to the query request body.

[0044] The service of the present application can be a query service. The present application designs a general data structure, and creates a service based on the data structure, so that the present application does not need to write SQL statements when creating a service. The present application directly generates the structured data of the service when creating the service.

[0045] In the embodiment of the present application, the structured data comprises: service ID, reference service ID, query type, query field list, table to be queried, query condition and sorting condition.

[0046] In the embodiment of the present application, the gateway will accept query requests from different channels (mobile terminal, Web terminal, other API calls), and route to the corresponding query engine (DataFetcher) according to the query request body (query Schema) in the input query request.

[0047] Different DataFetchers correspond to different data sources. The DataFetcher of the general data service will execute the construction of SQL according to the input query Schema data structure, perform syntax checking on the constructed SQL, and then return the data.

[0048] Step S102, generating the SQL statement of the service according to the structured data.

[0049] In an embodiment of the present application, the structured data can be parsed according to Druid SQL and an SQL statement can be constructed.

[0050] In an embodiment of the present application, the SQL statement is constructed, specifically including: constructing a field list, constructing a field list prefix, constructing a query source list, constructing a correlation condition, constructing a where condition, constructing a grouping field and condition, and constructing a sorting field.

[0051] Step S103, performing data query according to the SQL statement to obtain a data query result.

[0052] In an embodiment of the present application, the data query result is obtained by performing the SQL statement.

[0053] The query of the service of the present application is executed by a query engine, without developing different codes for different query execution, and the result of service orchestration, i.e., the service, is uniformly abstracted, and when the query is called, it is uniformly routed from the gateway to the query engine of the specific service, the corresponding service is queried by the executor of the engine, and a reverse construction of the SQL is performed on the corresponding service, the constructed SQL is transmitted to the JdbcTemplate query, and a unified query result is returned.

[0054] Figure 2 FIG. 2 is a second flowchart of the data query method of an embodiment of the present application, as shown in FIG. 2, in an embodiment of the present application, the service orchestration flow of the present application includes steps S201 to S203. Figure 2

[0055] Step S201, obtaining orchestration information of a service to be created, wherein the orchestration information is obtained by detecting the operation of a user on a service orchestration interface.

[0056] In an embodiment of the present application, the user performs service orchestration by performing module drag and combination on a visual service orchestration interface.

[0057] Step S202, generating structured data of the service to be created according to the orchestration information, and completing creation of the service.

[0058] Step S203, establishing a corresponding relationship between the completed created service and a query engine according to the domain of the completed created service.

[0059] In an embodiment of the present application, after the creation of the service is completed, the present application further includes:

[0060] ​The service call document of the created service is generated, so that a user makes a service query according to the service call document, wherein the service call document comprises a request address of the service, a query request body of the service, a return structure, optional parameters and mandatory parameters.

[0061] The service call of the application is routed through a gateway, and is routed to different query engines according to different fields corresponding to different services. After the service is published, the service call document can be generated, the service can be queried based on the GraphQL syntax to customize the data request format, flexible data query and aggregation query can be achieved, and the data source behind the different services can be diversified, can be from a relational database MySQL, an analytical database ClickHouse, an index database Elasticsearch, and a self-designed query engine. The query engine is routed to the corresponding data query source according to different services corresponding to different fields. The specific operation comprises the following steps:

[0062] The service is arranged through a visual platform, the user selects a data source in different fields, an index table in the field can be displayed, and then the data structure corresponding to the service can be formed by dragging the table, selecting a table field and filling a table condition.

[0063] The service is published according to the result arranged by the user in the previous step, and after the arrangement result is published, the service is mapped to different query engines according to different fields, and the query engines are responsible for unified query.

[0064] The service is called. For the arranged service, the query result can be viewed by simulating the call. When the arrangement result of the service is inconsistent with the expected effect, the service can be arranged, modified and published again. The graph information and the data structure information of the arrangement result of the service are saved each time, the information of the corresponding service is loaded when the service is modified next time, the interface document of the service is automatically generated before the service is called, and the user can dynamically construct a query Schema.

[0065] From the above embodiments, it can be seen that the application is based on the self-designed general orchestration data structure, abstracts SQL, uses Druid SQL for reverse construction and parsing of SQL, and through self-development of multiple functional modules such as gateway, engine, constructor and the like, performs one-stop link for orchestration and release of services and rapid authorization use. Finally, the rapid orchestration, rapid online and rapid use of services are realized, the inflexibility of the current service orchestration platform in orchestrating services is well solved, the development of redundant repeated codes by different applications is abstracted, a general data service structure is abstracted, users can quickly release their own services without understanding code logic, the services can be immediately verified and used, and the service limiting parameters can be customized, such as the maximum number of requests per second of the service, the TPS of the service, the QPS of the service, the access token of the service, the access permission of the service, the development of the query logic of the corresponding service by different applications is reduced, the efficiency of the rapid orchestration, release and online of the service is improved, and the service is more lightweight and free.

[0066] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0067] Based on the same inventive concept, the embodiments of the application also provide a data query device, which can be used to implement the data query method described in the above embodiments, as described in the following embodiments. Since the principle of solving problems of the data query device is similar to that of the data query method, the embodiments of the data query device can refer to the embodiments of the data query method, and the repeated parts will not be described herein. The term "unit" or "module" used below can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and is conceived.

[0068] Figure 3 is the first structural block diagram of the data query device of the embodiments of the application, as Figure 3 shown, in an embodiment of the application, the data query device of the embodiments of the application comprises:

[0069] a structured data determination module 1, configured to send a query request body of a user to a corresponding query engine, so that the query engine determines the structured data of a service corresponding to the query request body;

[0070] a SQL statement generation module 2, configured to generate a SQL statement of the service according to the structured data;

[0071] The data query module 3 is configured to query data according to the SQL statement and obtain a data query result.

[0072] Figure 4 is a second structural block diagram of the data query device according to an embodiment of the application, as shown in the figure, Figure 4 In an embodiment of the application, the data query device according to an embodiment of the application further comprises:

[0073] The orchestration information acquisition module 4 is configured to acquire orchestration information of a service to be created, wherein the orchestration information is obtained by detecting an operation of a user on a service orchestration interface.

[0074] The service creation module 5 is configured to generate structured data of the service to be created according to the orchestration information, and complete creation of the service.

[0075] The query engine association module 6 is configured to establish a corresponding relationship between the service completed in creation and a query engine according to a field of the service completed in creation.

[0076] In an embodiment of the application, the data query device according to an embodiment of the application further comprises:

[0077] The service calling document generation module is configured to generate a service calling document of the service completed in creation, so that a user performs service query according to the service calling document, wherein the service calling document comprises a request address of the service, a query request body of the service, a return structure, optional parameters and mandatory parameters.

[0078] In an embodiment of the application, the SQL statement generation module 2 is specifically configured to parse the structured data according to Druid SQL and construct a SQL statement.

[0079] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer device is also provided. As shown in the figure, Figure 5 The computer device comprises a memory, a processor, a communication interface and a communication bus, and a computer program capable of running on the processor is stored on the memory, and the processor implements the steps in the above-mentioned embodiment method when executing the computer program.

[0080] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, chips, or combinations thereof.

[0081] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and units, such as the corresponding program units in the above-mentioned method embodiments of the application. The processor executes various functions and work data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the methods in the above-mentioned method embodiments.

[0082] The memory can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function. The data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0083] The one or more units are stored in the memory, and when executed by the processor, the method in the above-mentioned embodiments is executed.

[0084] The above-mentioned computer device specific details can be understood by referring to the corresponding related description and effects in the above-mentioned embodiments, which will not be described here.

[0085] To achieve the above object, according to another aspect of the present application, there is further provided a computer readable storage medium storing a computer program, which, when executed on a computer processor, implements the steps of the above data query method. Those skilled in the art can understand that implementing all or part of the processes of the above embodiment method can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium, and when executed, can include the processes of the above embodiment method. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0086] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, or they can be respectively manufactured into individual integrated circuit modules, or multiple modules or steps among them can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0087] The above only describes the preferred embodiments of the present application and is not used to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data query method, characterized in that: include: Sending the user's query request body to a corresponding query engine, so that the query engine determines structured data of the service corresponding to the query request body; Generate an SQL statement for the service according to the structured data; Perform data query according to the SQL statement to obtain data query results; The method further includes: obtaining orchestration information of the service to be created, wherein the orchestration information is obtained by detecting user operations on a service orchestration interface, wherein the user operations on the service orchestration interface include dragging and dropping to combine; generating structured data of the service to be created based on the orchestration information to complete the creation of the service; and establishing a corresponding relationship between the created service and a query engine based on the domain of the created service; After the creation of the service is completed, the method further includes: Generate a service call document for the created service, so that the user can query the service according to the service call document, wherein the service call document includes: the service request address, the service query request body, the return structure, optional parameters and required parameters; The method further includes: after the service is orchestrated, checking the query results through simulated calls; when the service orchestration results are inconsistent with the expected effects, re-orchestrating, modifying and publishing the services, while saving the graph information and data structure information of the results of each service orchestration; and automatically generating a service interface document before the service is called.

2. The data query method according to claim 1, characterized in that: Generating the SQL statement of the service according to the structured data specifically includes: The structured data is parsed according to Druid SQL and an SQL statement is constructed.

3. The data query method according to claim 2, characterized in that: The constructing of the SQL statement specifically includes: constructing a field list, constructing a field list prefix, constructing a query source list, constructing an association condition, constructing a where condition, constructing grouping fields and conditions, and constructing a sorting field.

4. The data query method according to claim 1, characterized in that: The structured data includes: service ID, reference service ID, query type, query field list, table to be queried, query condition and sorting condition.

5. A data query device, characterized in that: include: a structured data determination module, configured to send the user's query request body to a corresponding query engine, so that the query engine determines the structured data of the service corresponding to the query request body; An SQL statement generating module, configured to generate an SQL statement for the service according to the structured data; A data query module is used to perform data query according to the SQL statement and obtain data query results; The apparatus further includes: an orchestration information acquisition module configured to acquire orchestration information of a service to be created, wherein the orchestration information is obtained by detecting user operations on a service orchestration interface; a service creation module configured to generate structured data of the service to be created based on the orchestration information, thereby completing service creation; and a query engine association module configured to establish a corresponding relationship between the created service and a query engine based on the domain of the created service. The device further includes: a service call document generation module, configured to generate a service call document for the created service, so that a user can query the service according to the service call document, wherein the service call document includes: a service request address, a service query request body, a return structure, optional parameters, and mandatory parameters; The device also includes: a simulation call module, which is used to view the query results through simulation calls after the service is arranged. When the service arrangement result is inconsistent with the expected effect, the service is re-arranged, modified and released, and the graph information and data structure information of the result of each service arrangement are saved; before the service is called, a service interface document is automatically generated.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed in a computer processor, the computer program implements the method according to any one of claims 1 to 4.

8. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images