Data query processing method, device and equipment, medium and data query system

Abstract

The invention provides a data query processing method, device and equipment, a medium and a data query system, and relates to the field of data processing, in particular to the technical field of cloud computing and intelligent products. According to the specific implementation scheme, a query statement is received; the query statement is sent by calling a standard service interface; detecting query parameters associated with the query statement and standard configuration information of each query parameter; determining a standard statement according to the standard configuration information of each query parameter and the query statement; sending the standard statement to an accessed database, so that the database executes the standard statement; and receiving a query result fed back by the database, and feeding back the query statement. According to the embodiment of the invention, the reusability of the data service can be improved, and the data repeated construction cost is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, specifically to the fields of cloud computing and intelligent product technologies, and particularly to a data query processing method, apparatus, data query processing device, non-transitory computer-readable storage medium, computer program product, and data query system. Background Art

[0002] In the field of enterprise data analysis, business personnel in an enterprise can integrate and analyze data of multiple business topics (such as sales, finance, supply chain, and human resources, etc.) through Business Intelligence (BI) tools without relying on the Information Technology (IT) department, to achieve cross-domain data-driven decision-making.

[0003] Currently, enterprise data analysis platforms generally adopt a mode of jointly building with multiple platforms and multiple data products, and can achieve multi-platform and multi-data associated and visual data analysis through interactive operations. Summary of the Invention

[0004] The present disclosure provides a data query processing method, apparatus, data query processing device, non-transitory computer-readable storage medium, computer program product, and data query system.

[0005] According to one aspect of the present disclosure, there is provided a data query processing method, including:

[0006] Receiving a query statement; the query statement is sent by calling a standard service interface;

[0007] Detecting query parameters associated with the query statement, and standard configuration information of each of the query parameters;

[0008] Determining a standard statement according to the standard configuration information of each of the query parameters and the query statement;

[0009] Sending the standard statement to an accessed database, so that the database executes the standard statement;

[0010] Receiving a query result fed back by the database, and giving a feedback for the query statement.

[0011] According to one aspect of the present disclosure, there is provided a data query processing apparatus, including:

[0012] A query statement receiving module, configured to receive a query statement; the query statement is sent by calling a standard service interface;

[0013] A parameter configuration acquisition module, configured to detect query parameters associated with the query statement and standard configuration information of each of the query parameters;

[0014] A standard statement determination module, configured to determine a standard statement according to the standard configuration information of each of the query parameters and the query statement;

[0015] A standard statement execution module, configured to send the standard statement to an accessed database so that the database executes the standard statement;

[0016] A query result receiving module, configured to receive a query result fed back by the database and give a feedback for the query statement.

[0017] According to another aspect of the present disclosure, there is provided a data query processing device, including:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the data query processing method according to any embodiment of the present disclosure.

[0021] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the data query processing method according to any embodiment of the present disclosure.

[0022] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, the data query processing method according to any embodiment of the present disclosure is implemented.

[0023] According to another aspect of the present disclosure, there is provided a data query system, including: a client, a data query processing device according to any embodiment of the present disclosure, and a database; the client is communicatively connected to the data query processing device, and the data query processing device is communicatively connected to the database;

[0024] The client is configured to send a query statement to the data query processing device;

[0025] The data query processing device is configured to generate a standard statement according to the query statement, send the standard statement to the database, and feed back a query result fed back by the database for the standard statement to the client;

[0026] The database is used to execute the standard statement, obtain the query result, and feedback the query result to the data query processing device.

[0027] The embodiments of the present disclosure can improve the reusability of data services and reduce the cost of duplicate data construction.

[0028] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0029] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0030] Figure 1 is a flowchart of a data query processing method disclosed according to an embodiment of the present disclosure;

[0031] Figure 2 is a flowchart of another data query processing method disclosed according to an embodiment of the present disclosure;

[0032] Figure 3 is a flowchart of another data query processing method disclosed according to an embodiment of the present disclosure;

[0033] Figure 4 is a scenario diagram of another data query processing method disclosed according to an embodiment of the present disclosure;

[0034] Figure 5 is a schematic structural diagram of a data query processing device disclosed according to an embodiment of the present disclosure;

[0035] Figure 6 is a block diagram of an electronic device for a data query processing method disclosed according to an embodiment of the present disclosure;

[0036] Figure 7 is a schematic structural diagram of a data query processing system disclosed according to an embodiment of the present disclosure;

[0037] Figure 8 is a scenario diagram of a data query processing system disclosed according to an embodiment of the present disclosure. Detailed Embodiments

[0038] The following describes exemplary embodiments of the present disclosure with reference to the drawings. Various details of the embodiments of the present disclosure are included to assist understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0039] Figure 1 It is a flowchart of a data query processing method disclosed according to an embodiment of the present disclosure. This embodiment can be applicable to the situation of receiving a query statement sent by any accessed client, performing corresponding queries on any accessed database, and receiving the query result and feedbacking it to the client. The method of this embodiment can be executed by a data query processing device, which can be implemented in a software and / or hardware manner and is specifically configured in an electronic device with certain data computing capabilities. The electronic device can be a server device.

[0040] S101. Receive a query statement; the query statement is sent by calling a standard service interface.

[0041] Among them, the query statement is used to call the database to execute and receive the query result feedback by the database. The query statement can be a statement sent by a sender that has established a communication connection with the data query processing device. The standard service interface can be an external service interface provided by the data query processing device. For example, the standard service interface can be an interface provided to the accessed client. The standard service interface can be an Application Programming Interface (API). The data query processing device can interface with any client or any client with permissions. The accessed client calls the standard service interface and passes the query statement into the standard service interface to enable the data query processing device to receive the query statement. In some embodiments, relative to the accessed client, the data query processing device can be understood as a database.

[0042] S102. Detect the query parameters associated with the query statement and the standard configuration information of each query parameter.

[0043] Among them, the query parameter can be a parameter calculated from the data in the database. Generally, the query parameter can be a parameter calculated from the data in the database according to at least one dimension. The query parameter associated with the query statement can refer to the parameter that needs to be aggregated and calculated as indicated by the query statement. The query statement can be parsed to obtain the object to be queried, and it is detected whether each queried object is within the preset parameter range. If the queried object is within the preset parameter range, the parameter corresponding to the queried object is determined as the query parameter associated with the query statement. The associated query parameter can be empty, or the number of associated query parameters is at least one. When the number of associated query parameters is at least one, the standard configuration information of the query parameter is obtained. The standard configuration information can refer to the configuration information of general parameters. Among them, the configuration information can include at least one of the following: the identifier of the parameter, the numerical type of the parameter, and the logical calculation formula of the parameter, etc. In some embodiments, the user can pre-configure a large number of parameters and generate the standard configuration information for each parameter. In one example, the logical calculation formula of the parameter can be: b = SUM(a), or b = MAX(a), or c = COUNT(a) / COUNT(MAX(b)).

[0044] S103. Determine the standard statement according to the standard configuration information of each of the query parameters and the query statement.

[0045] Among them, the standard statement can refer to a general database statement, and the standard statement is used to adapt to the database so that the database can be executed. According to the standard configuration information of each query parameter, it is detected whether the query statement needs to be rewritten; when the query statement needs to be rewritten, the query statement is rewritten according to the standard configuration information to generate the standard statement.

[0046] S104. Send the standard statement to the accessed database so that the database executes the standard statement.

[0047] Among them, the standard statement can refer to the statement parsed and executed by the database.

[0048] S105. Receive the query result fed back by the database and give feedback for the query statement.

[0049] Among them, the query result can refer to the result obtained by the database executing the standard statement, querying data and processing it. After receiving the query result, the query result is fed back to the client according to the information of the client passed in through the standard service interface.

[0050] According to the technical solution of the present disclosure, by receiving a query statement sent by a standard service interface, determining a standard statement according to the query parameters associated in the query statement, sending the standard statement to a database for execution, obtaining a query result fed back by the database, and feeding it back to the query party that sent the query statement, a standard service interface can be provided for the accessed query party to achieve data standardization, and a reusable data query service can be implemented according to the standard statement executable by the database for the query statement, thereby ensuring the consistency of data caliber, reducing errors caused by inconsistent processing logics, improving the accuracy of data processing, improving the reusability of data services at the same time, and reducing the cost of duplicate data construction.

[0051] In an optional embodiment, the receiving the query statement includes: receiving a query statement sent by a client, and the query statement is sent by calling a standard service interface through a visualization application configured in the client.

[0052] Among them, the query party accessing the data query processing device is the client. The visualization application of the client is used to obtain the visualization interaction operation of the user, receive the query statement, and send it to the data query processing device. The data query processing device and the client can be communicatively connected in a manner of a general database driver. In some embodiments, based on the general remote JDBC (Remote Java Database Connectivity) communication technology, Java application programs compatible with the BI function of the client can be accessed. The Java application program in the client remotely accesses the data query processing device through a network protocol.

[0053] In some embodiments, the client calls the standard service interface with information such as the resource location address (URL), username, and password of the database, and the data query processing device receives the resource location address (Uniform Resource Locator, URL), username, password, and query statement passed in by the standard service interface. The data query processing device can verify the client and the account (user) according to the database URL, username, and password. When the verification passes, the query statement is processed; when the verification fails, the query statement is discarded, and a response result of verification failure is fed back to the client, etc.

[0054] It can be seen that by calling the standard service interface through the visualization application of the client to send the query statement to the data query processing device, the operation complexity of the client can be reduced, thereby reducing the complexity of data query processing, facilitating self-service data analysis, and improving the efficiency of self-service data analysis.

[0055] Figure 2It is a flowchart of another data query processing method disclosed according to an embodiment of the present disclosure, which is further optimized and extended based on the above technical solution and can be combined with each of the above optional embodiments. The detecting the query parameters associated with the query statement and the standard configuration information of each query parameter is specifically: parsing the query statement to obtain at least one query object included in the query statement; obtaining the object parameters of the standard data model; in response to determining that there are object parameters corresponding to the query object, determining at least one query parameter associated with the query statement according to the existing object parameters corresponding to the query object; and obtaining the standard configuration information of each query parameter according to the standard data model.

[0056] S201. Receive a query statement; the query statement is sent by calling a standard service interface.

[0057] S202. Parse the query statement to obtain at least one query object included in the query statement.

[0058] Among them, the query object may refer to the object to be queried in the query statement. Parsing the query statement can obtain the object to be queried, the query type, the query structure, etc. described in the query statement. Obtain the object to be queried from the parsing result of the query statement and determine it as the query object.

[0059] S203. Obtain the object parameters of the standard data model.

[0060] Among them, the standard data model may refer to a data model with a unified standard semantics constructed in advance. Among them, this standard data model is used to configure the relationship between parameters and the metadata of the data at the physical level, and does not involve the specific content of the data at the physical level. A large number of object parameters are pre-configured in the standard data model, and there is a logical relationship between the object parameters and the data in the database. It is possible to detect whether the object parameters pre-configured in the standard data model correspond to each query object to detect the object parameters corresponding to the query object and obtain the parameter detection results of each query object. The parameter detection result of the query object may include that there are object parameters corresponding to the query object, or there are no object parameters corresponding to the query object. Among them, the object parameters corresponding to the query object may include: object parameters with the same attributes (such as identifiers) as the query object, and / or object parameters with the same description content as the query object.

[0061] S204. In response to determining that there are object parameters corresponding to the query object, determine at least one query parameter associated with the query statement according to the existing object parameters corresponding to the query object.

[0062] Among them, when there are object parameters corresponding to the query object, the existing object parameters corresponding to the query object can be determined as the query parameters associated with the query statement. When the query parameter is a derived parameter, the object parameters participating in the derived calculation of the query parameter can also be determined as the query parameters associated with the query statement. The query parameters associated with the query statement may refer to the object parameters required for querying and derived calculation when executing the query statement. The query parameters associated with the query statement may include: the object parameters corresponding to the query object, and / or the object parameters participating in the derived calculation of the determined query parameters.

[0063] S205. Obtain the standard configuration information of each of the query parameters according to the standard data model.

[0064] Among them, the standard configuration information may be the configuration information of the query parameters in the parameter standard data model. When configuring parameters in the standard data model, the standard configuration information of the parameters is set simultaneously. The standard configuration information may include: the identifier of the query parameter, the description content, the attributes, the logical calculation formula, etc.

[0065] S206. Determine the standard statement according to the standard configuration information of each of the query parameters and the query statement.

[0066] Among them, the standard statement includes at least one query field, and the query field corresponds to the field (key) existing in the database. That is, the query field is the information that the database can support parsing and processing. The conversion from the query statement to the standard statement is equivalent to the conversion from the query object that the database cannot parse and process to the query field that the database can parse and process.

[0067] S207. Send the standard statement to the accessed database so that the database executes the standard statement.

[0068] S208. Receive the query result fed back by the database and give feedback on the query statement.

[0069] According to the technical solution of the present disclosure, by configuring query parameters and the standard configuration information of the query parameters in the standard data model, determining the standard statement based on the standard configuration information of the query parameters, and providing it for the database to execute, a standard and consistent data relationship can be established for the database, realizing unified metadata management, reducing redundant structures, improving query efficiency, enhancing data consistency, improving data query accuracy. The standard and consistent data relationship can improve the reusability of data services and reduce the cost of repeated data construction.

[0070] In an optional embodiment, the standard data model includes: a data table model; the data table model is generated by the following steps: obtaining the fields included in at least one data table in the accessed database; establishing the mapping relationships between the data tables according to the fields included in each data table; generating a data table model according to the mapping relationships between the data tables and the fields included in each data table, and using it as the standard data model.

[0071] Among them, the data table model is used to determine the metadata of the data at the physical level and the relationships between the data at the physical level determined according to the metadata. Among them, the metadata may include the fields (identifiers), the attributes of each field, and the description content of each field in at least one data table in the accessed database. In some embodiments, the data table model includes the fields included in the data tables in the accessed database and the fields with mapping relationships between different data tables. The data table model may include the metadata of all the data tables in the accessed database or the metadata of some data tables in the accessed database. The same fields in different data tables can be determined according to the identifiers and description content of the fields in different data tables, and the mapping relationships between at least two data tables with the same fields can be established based on the same fields. Among them, the field may be a key, and the mapping relationship of the foreign keys of different data tables can be established.

[0072] In some embodiments, a graph database can be constructed, taking one data table as an element, and adding the metadata of the data table as additional information of the element to the element of the data table. The elements of at least two data tables to which the fields with mapping relationships belong can be connected, and the fields with mapping relationships can be added as additional information of the connection line to the line.

[0073] It can be seen that by generating a data table model from the fields included in each data table and the mapping relationships between the fields included in each data table, the data in the multi-source heterogeneous data tables can be mapped into a data table model with unified semantics. Based on the data table model with unified semantics for data query, the complexity of business query can be reduced, and data tables with different structures can all be mapped using the same data table model, realizing data reuse and standardization construction, and reducing the cost of repeated data research and development.

[0074] In an optional embodiment, the step of sending the standard statement to the accessed database to enable the database to execute the standard statement includes: obtaining the query range corresponding to the standard statement; sending the standard statement and the corresponding query range to the accessed database to enable the database to execute the standard statement within the corresponding query range.

[0075] Among them, the query range can refer to the range in the storage space where the query operation is performed in the database. The query range corresponding to the standard statement can be determined based on at least one of the query object in the query statement, the fields involved in the logical calculation formula of the query parameters, and the query fields in the standard statement. For each item, the corresponding range can be determined, and the union of the determined ranges is taken to obtain the query range corresponding to the standard statement. The query range is used by the database to narrow the query range and avoid querying the entire data table.

[0076] Among them, the standard statement and the corresponding query range can be directly sent to the accessed database; or, the query ranges corresponding to each query object can be added to the standard statement respectively, and the added standard statement can be sent to the accessed database, which is equivalent to sending the standard statement and the corresponding query range to the accessed database.

[0077] It can be seen that by obtaining the query range corresponding to the standard statement and sending it to the database, so that the database performs the data query operation corresponding to the standard statement within the query range, the number of queries can be reduced and the query efficiency can be improved.

[0078] In an optional embodiment, the obtaining the query range corresponding to the standard statement includes: screening the data tables corresponding to the standard statement in each data table according to each query field in the standard statement and the fields included in each data table; and determining the data table corresponding to the standard statement as the query range corresponding to the standard statement.

[0079] Among them, the query field in the standard statement can refer to the object to be queried in the standard statement. The data table corresponding to the standard statement can refer to the data table including the values corresponding to the query fields in the standard statement. The number of data tables corresponding to the standard statement is at least one. According to the query fields in the standard statement, queries can be made in the fields included in each data table, and the data tables to which at least one field identical to the query field belongs are determined as the data tables to which the query field belongs. The data table to which the query field belongs is determined as the query range corresponding to the query field. The query ranges corresponding to each query field in the standard statement are determined as the query range corresponding to the standard statement. Obtaining the query range corresponding to the standard statement is equivalent to pruning the data table model and removing the data tables irrelevant to the standard statement.

[0080] It can be seen that by the query fields in the standard statement and the fields included in the data table, the data table corresponding to the standard statement is determined and used as the query range corresponding to the standard statement, which can narrow the data tables to be queried, achieve accurate positioning of the query range, and improve the query efficiency.

[0081] In an alternative embodiment, after determining the data table corresponding to the standard statement as the query range corresponding to the standard statement, the following steps are further included: generating traceability data for the query result based on the standard statement and the corresponding data table; and providing feedback for the query statement based on the traceability data.

[0082] Among them, the traceability data of the query result may refer to the source and changes of the query result, and is used to determine the flow direction and processing process, etc. during the transfer process of the data involved in the query result. Taking the query result as the end point, tracing the metadata and their relationships related to the query result, and / or taking the query fields in the standard statement as the starting point, tracing all the relevant metadata and their relationships of each query field, and generating a graph database with the data tables to which the traced metadata belongs and the relationships between the data tables, and using it as the traceability data to provide feedback for the query statement. Among them, the traced data table is the data table corresponding to the standard statement.

[0083] In some embodiments, taking a data table as an element according to the query fields in the standard statement, the query order of the query fields, and the data table corresponding to the query fields, sorting the elements of the data table in the query order, connecting the elements of the data tables to which the query fields with dependency relationships belong, and arranging the elements of the data tables of the query fields without dependency relationships in parallel, and generating a graph database with the sorted elements and the connection relationships between the elements as the traceability data.

[0084] It can be seen that by generating traceability data based on the standard statement and the data table corresponding to the standard statement, and providing the traceability data to the query party as well, the traceability of the query result can be realized, which is convenient for tracing the source and processing process of the query result and facilitating the positioning of data processing anomalies.

[0085] In an alternative embodiment, the standard data model includes: a parameter model; the parameter model is generated in the following manner: receiving a parameter configuration request; generating object parameters according to the parameter configuration request, and the standard configuration information corresponding to the generated object parameters; and adding the generated object parameters and the standard configuration information corresponding to the generated object parameters to the parameter model.

[0086] Among them, the parameter model is used to determine the logical calculation relationship between the metadata in the database and the object parameters. Specifically, the parameter model is used to determine the logical calculation relationship between the metadata in the data table model and the object parameters. A client with permissions can configure the object parameters. The parameter configuration request is used to provide relevant information about the object parameters to generate the object parameters. From the parameter configuration request, the identifier of the object parameter is extracted, and the object parameter is generated. Also, from the parameter configuration request, the logical calculation formula of the object parameter is extracted to generate the standard configuration information of the object parameter. The parameter model may include at least one object parameter and the standard configuration information of each object parameter.

[0087] It can be seen that by generating the object parameters and the corresponding standard configuration information based on the parameter configuration request and adding them to the parameter model, the logical calculation relationship between the metadata in the database and the object parameters is realized. This can establish a standard and consistent data relationship for the database, align the parameter requirements of different clients to the same database, adapt to diverse business parameter requirements, improve the flexibility of the parameters, and thus improve the flexibility and diversity of the query content. By establishing the relationship between any parameter and the metadata in the database with any data structure, the consistency of the data caliber can be ensured and the accuracy of data query can be improved.

[0088] Figure 3 It is a flowchart of another data query processing method disclosed according to an embodiment of the present disclosure, which is further optimized and extended based on the above technical solution and can be combined with each of the above optional embodiments. The determination of the standard statement according to the standard configuration information of each query parameter and the query statement is specifically: according to each query parameter, determine whether to rewrite the query statement to obtain a rewrite detection result; in response to determining that the rewrite detection result indicates a rewrite, according to the standard configuration information of each query parameter, detect the parameter type of each query parameter; rewrite the query statement according to each parameter type to obtain the standard statement; in response to determining that the rewrite detection result indicates to keep the original sentence, determine the query statement as the standard statement.

[0089] S301. Receive a query statement; the query statement is sent by calling a standard service interface.

[0090] S302. Detect the query parameters associated with the query statement and the standard configuration information of each query parameter.

[0091] S303. According to each query parameter, determine whether to rewrite the query statement to obtain a rewrite detection result.

[0092] Among them, the rewriting detection result indicates rewriting or keeping the original sentence. Keeping the original sentence is equivalent to not rewriting the query statement. When the query parameter is empty, it is determined that the rewriting detection result is to keep the original sentence. The rewriting detection result can be determined according to the number of query parameters. For example, when the number of query parameters is zero, it is determined that the rewriting detection result indicates keeping the original sentence; when the number of query parameters is non-zero, it is determined that the rewriting detection result indicates rewriting. Or the rewriting detection result can be determined according to the information associated with the query parameters. For example, according to the information associated with each query parameter, it is determined that each query parameter is the field itself in the data table, and it is determined that the rewriting detection result indicates keeping the original sentence; when at least one query parameter is obtained by aggregating fields in the data table, it is determined that the rewriting detection result indicates rewriting.

[0093] S304. In response to determining that the rewriting detection result indicates rewriting, according to the standard configuration information of each of the query parameters, detect the parameter type of each of the query parameters.

[0094] Among them, the parameter type is used to describe the aggregation complexity of the query parameter. The aggregation complexity of the query parameter can be determined according to the logical calculation formula in the standard configuration information, and the parameter type of the query parameter can be determined according to the aggregation complexity. In some embodiments, the aggregation complexity can be determined according to whether the logical calculation formula includes aggregation calculation and the number of fields or parameters participating in the aggregation calculation. In some embodiments, the parameter type of the query parameter can be determined according to the corresponding relationship between the aggregation complexity and the parameter type.

[0095] S305. Rewrite the query statement according to each of the parameter types to obtain a standard statement.

[0096] Among them, different parameter types correspond to different statement rewriting methods. The query statement is rewritten using the statement rewriting method corresponding to the parameter type to obtain a standard statement. In fact, rewriting the query statement is equivalent to rewriting the custom query parameters in the query statement into the content of the fields recognizable and processable by the database. The query parameter corresponds to the parameter type one by one. The query parameter can be rewritten according to the parameter type of a query parameter. Rewriting the query statement can be to rewrite each query parameter in the query statement respectively. After each query parameter in the query statement is rewritten, a standard statement is obtained.

[0097] S306. In response to determining that the rewriting detection result indicates keeping the original sentence, determine the query statement as the standard statement.

[0098] Among them, keeping the original sentence means that there is no need to rewrite the query statement, and the query statement can be directly determined as the standard statement. In some embodiments, when each query parameter is empty, that is, there is no object parameter corresponding to the query object, it is determined that the rewriting detection result indicates keeping the original sentence.

[0099] S307. Send the standard statement to the accessed database so that the database executes the standard statement.

[0100] S308. Receive the query result fed back by the database and give feedback on the query statement.

[0101] According to the technical solution of the present disclosure, by detecting whether the query statement needs to be rewritten according to the query parameters, and when it is detected that rewriting is required, rewriting the query statement according to the parameter type determined by the standard configuration information to obtain a standard statement, the query statement can be rewritten for flexible standard configuration information, can adapt to flexible and diverse parameter configuration information, improve the accuracy of processing the query requirements of different query parties, and reduce the implementation cost of service parameters.

[0102] In an optional embodiment, detecting the parameter type of each query parameter according to the standard configuration information of each query parameter includes: in response to determining that the standard configuration information includes a non-aggregation calculation formula, determining that the parameter type is a replacement type; in response to determining that the standard configuration information includes a low-aggregation calculation formula, determining that the parameter type is a nested type; in response to determining that the standard configuration information includes a high-aggregation calculation formula, determining that the parameter type is a fusion type.

[0103] Among them, the standard configuration information including a non-aggregation calculation formula may mean that the logical calculation formula in the standard configuration information only includes non-aggregation calculation logic, where the non-aggregation calculation logic may be non-aggregation calculation logic of any number of any types (parameters or fields), that is, the non-aggregation calculation formula does not include any aggregation calculation logic. The replacement type may mean simply replacing the parameter type of the logical calculation formula of the query parameter.

[0104] The standard configuration information includes a low-aggregation calculation formula, which can refer to that the logical calculation formula in the standard configuration information includes the calculation logic of the aggregation of one parameter or the aggregation calculation logic of any number of fields. When the low-aggregation calculation formula includes the calculation logic of the aggregation of one parameter, the low-aggregation calculation formula can also include the aggregation calculation logic of any number of fields. In addition, the low-aggregation calculation formula can also include the non-aggregation calculation logic of any number of any types (parameters or fields). The calculation logic of the aggregation of one parameter can refer to that the standard configuration information of the parameter includes the aggregation calculation logic, or the parameter can be directly called an aggregation parameter. Among them, the logical calculation formula of parameter A includes the calculation logic of parameter B, and parameter A is a derived parameter. The logical calculation formula of parameter A does not include the calculation logic of any parameter, and parameter A is not a derived parameter. Correspondingly, when the query parameter is a derived parameter and the number of parameters in the aggregation calculation logic of the parameters included in the logical calculation formula of the query parameter is less than or equal to one, it is determined that the standard configuration information of the query parameter includes a low-aggregation calculation formula. When the query parameter is not a derived parameter and the logical calculation formula of the query parameter includes the aggregation calculation logic of fields, it is determined that the standard configuration information of the query parameter includes a low-aggregation calculation formula. The nested type can refer to the type of nesting the logical calculation formula of a parameter into a parameter.

[0105] The standard configuration information includes a high-aggregation calculation formula, which can be that the logical calculation formula in the parameter standard configuration information includes the calculation logic of the aggregation of at least two parameters. In addition, the high-aggregation calculation formula can also include the non-aggregation calculation logic of any number of any types (parameters or fields). The high-aggregation calculation formula is the logical calculation formula of a derived parameter. The fusion type can refer to the type of fusing the logical calculation formula of a parameter and nesting parameters.

[0106] Sorted by the aggregation complexity, the non-aggregation calculation logic and the replacement type have the lowest corresponding aggregation complexity, the low-aggregation calculation formula and the nested type have a medium corresponding aggregation complexity, and the high-aggregation calculation formula and the fusion type have the highest aggregation complexity.

[0107] In one example, for all query parameters included in the query statement, when a query parameter does not require aggregation calculation, it is determined that the logical calculation formula of the query parameter only includes non-aggregation calculation logic, so as to determine that the parameter type of the query parameter is the replacement type, where the query parameter can be a derived parameter or not. For another example, when the logical calculation formula of the query parameter only includes the aggregation calculation logic of fields, it is determined that the logical calculation formula of the query parameter includes a low-aggregation calculation formula, so as to determine that the parameter type of the query parameter is the nested type. For another example, when the query parameter is a derived parameter and the logical calculation formula of the query parameter includes the aggregation calculation logic of one parameter, or when the query parameter is a derived parameter and the logical calculation formula of the query parameter only includes the aggregation calculation logic of fields, it is determined that the logical calculation formula of the query parameter includes a low-aggregation calculation formula, so as to determine that the parameter type of the query parameter is the nested type. For another example, when the query parameter is a derived parameter and the logical calculation formula of the query parameter includes the aggregation calculation logic of at least two parameters, it is determined that the logical calculation formula of the query parameter includes a high-aggregation calculation formula, so as to determine that the parameter type of the query parameter is the fusion type.

[0108] It can be seen that by determining the parameter type according to the information of the aggregation calculation formula included in the standard configuration information, and realizing the rewriting method of the query parameter based on the aggregation complexity of the logical calculation formula of the parameter, it is possible to adapt to flexible parameter logical calculation formulas, corresponding to the rewriting method of the parameter, and improve the accuracy of parameter rewriting.

[0109] In an optional embodiment, the rewriting the query statement according to each of the parameter types to obtain a standard statement includes: in response to determining that the parameter type is the replacement type, determining replacement data for each of the query parameters according to the standard configuration information; replacing the corresponding content in the query statement with the replacement data for each of the query parameters to obtain a standard statement.

[0110] Among them, the replacement data may refer to the data used to replace the query parameter in the query statement. The replacement data of the query parameter may refer to the logical calculation expression of the query parameter in the logical calculation formula. The logical calculation formula of the query parameter can be obtained from the standard configuration information, and the logical calculation expression of the query parameter can be determined, and used as the replacement data of the query parameter. The corresponding content in the query statement may refer to the identification content of the query parameter. The query parameter is directly replaced with the corresponding replacement data in the query statement, and after all query parameters are replaced, the replaced query statement is determined as the standard statement.

[0111] In one example, the query parameter is: the maximum number of employees in a department; the logical calculation formula for this query parameter is:

Maximum number of employees in a department

Maximum number of employees in a department

[0112] The Structured Query Language (SQL) query statement sent by the querying party's application layer is as follows:

[0113] Select a.Monthly as col0,a.Organization as col1, Maximum number of employees in a department as col2

[0114] From(

[0115] Select * from model

[0116] )a group by col0,col1

[0117] The above SQL means: Read all data from the table model, group by

Monthly

Organization

[0118] Among them,

Monthly

Organization

Number of employees in the department

Maximum number of employees in a department

[0119] The query statement after replacement, that is, the standard statement, is as follows:

[0120] Select a.Monthly as col0,a.Organization as col1, MAX(Number of employees in the department) as col2

[0121] From(

[0122] Select * from model

[0123] )a group by col0,col1

[0124] The above SQL statement means: Read all data from the table model, group by

Monthly

Organization

Number of Employees on the Job in the Department

[0125] It can be seen that for the replacement type, according to the standard configuration information of the query parameters, the replacement data of the query parameters can be determined, and the replacement data is used to replace the query parameters in the query statement, so as to rewrite the query statement of the replacement type parameters, improving the rewriting efficiency and accuracy of the query statement of the replacement type.

[0126] In an optional embodiment, the rewriting of the query statement according to each of the parameter types to obtain a standard statement includes: in response to determining that the parameter type is a nested type, determining the statements of each of the query parameters according to the standard configuration information of each of the query parameters; nesting the statements of each of the query parameters into the query statement to obtain the standard statement.

[0127] Among them, the statement of a query parameter can refer to the SQL statement of that query parameter. The statement of the query parameter can be added as a subquery to the parent query with the query statement as the parent query, so as to realize nesting the statement of the query parameter into the query statement. After the statements of all query parameters are nested, the nested query statement is determined as the standard statement.

[0128] In an example, the query parameter: Number of Employees on the Job in the Month of 24; the logical calculation formula of this query parameter is:

[0129] COUNT_AGG(Employee ID, [Monthly], Year = ‘2024’)

[0130] This formula means that

Number of Employees on the Job in the Month of 24

Monthly

Number of Employees on the Job in the Month of 24

[0131] The SQL of the structured query language requested by the querying party's application layer is:

[0132] Select a.Monthly as col0, a.Organization as col1, SUM(a.Number of Employees on the Job in the Month of 24) as col2

[0133] From(

[0134] Select*from model

[0135] )a group by col0,col1

[0136] The above SQL statement means: Read all data from the table'model', group by ['Monthly'] and ['Organization'], calculate the sum of the ['Number of Employees on the Job in 24 Months'] in each group, and name the result 'col2'.

[0137] Among them, ['Monthly'] and ['Organization'] are fields in the database, and ['Number of Employees on the Job in 24 Months'] is a query parameter. The logical calculation formula of ['Number of Employees on the Job in 24 Months'] includes the aggregation calculation logic of fields. The logical calculation formula of ['Number of Employees on the Job in 24 Months'] belongs to a low-aggregation calculation formula, and the parameter type is determined to be a nested type. Generate the SQL for ['Number of Employees on the Job in 24 Months']. Nest the SQL of ['Number of Employees on the Job in 24 Months'] into the position of ['Number of Employees on the Job in 24 Months'] in the query statement.

[0138] Specifically, the SQL generated according to the low-aggregation calculation formula COUNT_AGG(Employee ID, [Monthly], Year = '2024') is as follows:

[0139] Select agg_list,count(Employee ID)from table_filled where Year = '2024' group by agg_list

[0140] This SQL statement means: Group by the specified field 'agg_list' and count the number of employees in each 'agg_list' group in the table 'table_filled' in 2024.

[0141] The nesting method is: Replace 'agg_list' with the dimension ['Monthly'] and the dimension ['Organization']. Replace 'table_filled' in the above-generated SQL with the 'from' of the query statement SQL, and rewrite the SQL by using the generated SQL as the 'from' field of the query statement SQL.

[0142] The nested query statement, that is, the standard statement, is as follows:

[0143] Select a.Monthly as col0,a.Organization as col1, SUM(a.Number of Employees on the Job in 24 Months) as col2

[0144] From(

[0145] Select T.Monthly, T.Organization, count(Employee ID)

[0146] From(

[0147] Select*from model

[0148] )T

[0149] where year = '2024' group by month, organization

[0150] ) a group by col0, col1

[0151] The above SQL statement means: Read all data from table model, group by [month] and [organization], first count the number of employees in 2024 in each group as the [number of employees on the job in 24 months] of that group, then count the sum of the [number of employees on the job in 24 months] of each group, and name the result col2.

[0152] It can be seen that for nested types, according to the standard configuration information of the query parameters, the statement of the query parameters can be determined, and the statement of the query parameters can be nested into the query statement to rewrite the query statement of the parameters of the nested type, improving the rewriting efficiency and accuracy of the query statement of the nested type.

[0153] In an optional embodiment, the rewriting of the query statement according to each of the parameter types to obtain a standard statement includes: in response to determining that the parameter type is a fusion type, determining the statements of each of the query parameters and the statements of the sub-parameters in the query parameters according to the standard configuration information of each of the query parameters; performing dimension fusion on the statements of each sub-parameter in each of the query parameters to obtain the sub-statements of each of the query parameters; nesting the sub-statements of each of the query parameters into the statements of each of the query parameters to obtain the fusion statements of each of the query parameters; nesting the fusion statements of each of the query parameters into the statements of the query parameters; and nesting the fusion statement into the query statement to obtain the standard statement.

[0154] Among them, the query parameter with the parameter type of fusion type is a derived parameter. The parameters included in the logical calculation formula of the query parameter are the sub-parameters of the query parameter. The query parameter of the fusion type is a derived parameter, and the number of sub-parameters included in the derived parameter is at least two. Each sub-parameter has standard configuration information and a logical calculation formula. The statement of the sub-parameter in the query parameter may refer to the SQL statement of the sub-parameter. Dimension fusion may refer to fusing SQL statements according to the query dimension. For each query parameter, first fuse the statements of each sub-parameter in the query parameter to obtain the sub-statement of the query parameter, then nest the sub-statement into the statement of the query parameter to obtain the fusion statement of the query parameter, and finally add the fusion statement of the query parameter as a sub-query to the parent query with the query statement as the parent query to obtain the standard statement.

[0155] In one example, the query parameter: monthly turnover rate in 2024; the logical calculation formula for this query parameter is: monthly turnover rate in 2024 = number of monthly leavers / total number of employees in 2024. The derived parameter [turnover rate] includes two sub-parameters [number of monthly leavers] and [total number of employees in 2024].

[0156] Among them, the sub-parameter [number of monthly leavers]:

[0157] number of monthly leavers = COUNT_AGG(leaving employee ID, [monthly], year = '2024')

[0158] The sub-parameter [number of monthly leavers] represents the total number of all leaving employees in 2024, which is accumulated and calculated by [monthly].

[0159] The sub-parameter [total number of employees in 2024]: total number of employees in 2024 = COUNT_AGG(employee ID, Fixed[], year = '2024'))

[0160] The sub-parameter [total number of employees in 2024] represents the total number of employees in 2024 filtered out, without aggregation according to any dimension.

[0161] The SQL of the structured query language requested by the querying party's application layer is:

[0162] Select a.monthly as col0, a.organization as col1, monthly turnover rate in 2024 as col2

[0163] From(

[0164] Select * from model

[0165] ) a group by col0, col1

[0166] The above SQL means: Read all data from the table model, group by [monthly] and [organization], and count the [monthly turnover rate in 2024] for each group, with the result named col2.

[0167] 1. Generate SQL according to the logical calculation formula of the query parameter [turnover rate]:

[0168] Select monthly, organization, number of leaving employees / total number of employees as monthly turnover in 2024

[0169] From(

[0170] Select * from model

[0171] ) a group by agg_list

[0172] The above SQL statement means: Read all the data from the table model, group it by agg_list, calculate the result of the number of departing employees / total number of employees in each group, and name the result as

Monthly Departure Rate in 2024

[0173] Generate SQL according to the logical calculation formula of the sub-parameter

Monthly Departure Number

Total Number of Employees in 2024

Monthly Departure Number

Monthly

Total Number of Employees in 2024

[0174] 2. You can preferentially generate SQL according to the logical calculation formula of the sub-parameter

Monthly Departure Number

[0175] Select Monthly, Organization, count(Departing Employee ID) from table_filled where Year = ‘2024’ group by Monthly, Organization

[0176] The above SQL statement means: Read all the data in 2024 from the table table_filled, group it by

Monthly

Organization

[0177] 3. Then generate SQL according to the logical calculation formula of the sub-parameter

Total Number of Employees in 2024

[0178] Select Monthly, Organization, count(Departing Employee ID) as Number of Departing Employees

[0179] From table_filled where Year = ‘2024’

[0180] group by Monthly, Organization

[0181] ) t1 join ([

[0182] Select count(Employee ID) as Total Number of Employees from table_filled where Year = ‘2024’

[0183] ) t2 on 1 = 1

[0184] The above SQL statement means: Read all the data for the year 2024 from the table table_filled, group by

Monthly

Organization

[0185] 4. Nest the result of the SQL join into the SQL for the query parameter

Separation Rate

[0186] Select Monthly, Organization, number of departed employees / total number of employees as Monthly Separation Rate in 2024

[0187] From(

[0188] Select Monthly, Organization, count(departed employee ID) as number of departed employees

[0189] From(

[0190] select * from model

[0191] ) where year = ‘2024’

[0192] group by Monthly, Organization

[0193] ) t1 join(

[0194] Select count(employee ID) as total number of employees

[0195] from(

[0196] select * from model

[0197] ) where year = ‘2024’

[0198] ) t2 on 1 = 1

[0199] The above SQL statement means: Read all the data for the year 2024 from the table table_filled, group by

Monthly

Organization

Monthly Separation Rate in 2024

[0200] 5. The standard statement obtained by nesting the fusion statement into the query statement is as follows:

[0201] Select a.monthly as col0, a.organization as col1, monthly turnover rate in 2024 as col2

[0202] From(

[0203] Select monthly, organization, number of departing employees / total number of employees as monthly turnover rate in 2024

[0204] From(

[0205] Select monthly, organization, count(departing employee ID) as number of departing employees

[0206] From(

[0207] select * from model

[0208] ) where year = '2024'

[0209] group by monthly, organization

[0210] ) t1 join(

[0211] Select count(employee ID) as total number of employees

[0212] from(

[0213] select * from model

[0214] ) where year = '2024'

[0215] ) t2 on 1 = 1

[0216] ) a group by col0, col1

[0217] The above SQL statement means: Read all the data in 2024 from the table table_filled, group by [monthly] and [organization], first count the number of departing employees in each group and name it as the number of departing employees. Then read all the data in 2024 from the table table_filled, count the total number of employees in 2024 in each group, name the result as the total number of employees. Calculate the result of the number of departing employees / total number of employees, name the result as [monthly turnover rate in 2024], and finally name the monthly turnover rate in 2024 as col2.

[0218] It can be seen that for the fusion type, according to the standard configuration information of the query parameters, the statement of the query parameters can be determined, and the statement of the query parameters can be fused and nested into the query statement, so as to realize the rewriting of the query statement of the parameters of the fusion type and improve the rewriting efficiency and accuracy of the query statement of the fusion type.

[0219] In an optional embodiment, the parameter types of the query parameters in the query statement may include at least one of a replacement type, a nested type, and a fusion type. For each query parameter, according to the rewriting method corresponding to the parameter type, each query parameter can be rewritten accordingly, and the query statement after the rewriting of each query parameter is determined as the standard statement.

[0220] In an optional embodiment, the rewriting the query statement according to each parameter type to obtain a standard statement includes: obtaining the syntax tree of the query statement; updating the syntax tree of the query statement according to the parameter types and standard configuration information of the query parameters; and generating a standard statement according to the updated syntax tree.

[0221] Among them, when a query statement is received, the query statement can be parsed to generate the syntax tree of the query statement. Among them, the syntax tree is an Abstract Syntax Tree (AST), and the syntax tree is used to describe the query object and the statement structure in the query statement. When detecting whether there are parameters in the query statement, it can be detected whether there are parameters in the syntax tree by traversing the syntax tree of the query statement. According to the parameter types and standard configuration information of the query parameters, the corresponding rewriting methods for the query parameters are determined, and according to the corresponding rewriting methods, the logical calculation formulas of the query parameters are added to the syntax tree of the query statement. After all the logical calculation formulas of the query parameters are added, that is, when the syntax tree is updated, the updated syntax tree is converted into an SQL statement, and the SQL statement is determined as the standard statement.

[0222] In some embodiments, for the query parameters of the replacement type, the syntax tree can be updated according to the replacement data of the query parameters. For the query parameters of the nested type, the syntax tree can be updated according to the statement of the query parameters. For the query parameters of the fusion type, the syntax tree can be updated according to the statement of the query parameters and the statements of each sub-parameter in the query parameters. Specifically, the fusion statement of the query parameters can be determined according to the statement of the query parameters and the statements of each sub-parameter in the query parameters, and then the syntax tree can be updated according to the fusion statement. Updating the syntax tree can be adding, deleting, or modifying the nodes of the syntax tree and adding, deleting, or modifying the structures of the corresponding nodes.

[0223] It can be seen that by converting the query statement into a syntax tree and updating the syntax tree according to the parameter types of each query parameter and the standard configuration information, the query statement can be accurately rewritten to optimize the query statement, while having better compatibility and scalability.

[0224] Figure 4 It is a scenario diagram of a data query processing method disclosed according to an embodiment of the present disclosure.

[0225] As Figure 4 shown, among which, the client in the application layer provides services such as data pulling, data analysis, and dashboards. The client in the application layer sends a query statement. In the data query service of the data query processing device in the model service layer, the combined model proxy receives the query statement and sends it to the parameter service module in the data query service. The parameter service module receives the query statement, and the query statement is:

[0226] # Where the parameter formula for [Monthly Active Employees] is defined as: COUNT_AGG(Employee ID, [Monthly], Year = '2024')

[0227] # The requested SQL is:

[0228] Select a.Monthly as col0, a.Organization as col1,

[0229] Sum(a.Monthly Active Employees) as col2

[0230] From (

[0232] Select * from model

[0233] ) a group by a.Monthly, a.Organization

[0234] The parameter service module parses the query statement and generates a syntax tree.

[0235] The parameter service module traverses the syntax tree:

[0236] # Obtain the objects to be queried

[0237] [a.Monthly, a, Organization, a.Monthly Active Employees]

[0238] The parameter service module obtains the logical calculation formula of the query parameter and parses it:

[0239] # Generate SQL fragments for the formula

[0240] a.Monthly -> b.Monthly

[0241] a.Organization -> b.Organization

[0242] a. Monthly on-the-job personnel ->

[0243] {

[0244] Select: b.S1

[0245] From: ([[]]

[0246] Select: [count(a.Employee ID), a.Monthly, a.Organization]

[0247] Form: a

[0248] Where: a.Year = '2024'

[0249] Group by [a.Monthly, a.Organization]

[0250] alias: b )

[0252] The above SQL means: Set the alias of a.Monthly as b.Monthly, and the alias of a.Organization as b.Organization; Set the alias of a.Monthly on-the-job personnel as b.S1. b.S1 means reading all data in 2024 from table a, grouping by [Monthly] and [Organization], and counting the sum of employees in each group as b.S1.

[0253] The parameter service module backfills the generated SQL fragment into the syntax tree.

[0254] Based on the backfilled syntax tree, generate the standard statement:

[0255] # Output the final SQL to be executed

[0256] Select b.Monthly as col0 b.Organization as col1

[0257] Sum(b.S1) as col2

[0258] From (

[0260] Select count(a.Employee ID) as S1, a.Monthly, a.Organization

[0261] Form: (

[0262] Select * from model

[0263] ) a

[0264] Where: a.Year = ‘2024’

[0265] Group by a. Monthly, a, Organization

[0266] )b Group by b. Monthly, b. Organization

[0267] }

[0268] The above SQL statement means: Read all data in table a in 2024, group by [a. Monthly] and [a. Organization], count the sum of employees in each group, name the result S1, read all data in table b, group by [b. Monthly] and [b. Organization], count the sum of S1 in each group, and name the result col2.

[0269] The data query service prunes the data table model according to the query fields in the standard statement, determines the query range based on the pruned data table model, sends the query range and the standard statement to the database at the physical layer, and the database executes the standard statement and returns the query result. The data query service returns the feedback query result to the client at the application layer.

[0270] The embodiments of the present disclosure ensure the consistency of data caliber and improve the accuracy of analysis conclusions by unifying the data model and parameter system; provide flexible parameter definition methods and cross-topic analysis capabilities, and reduce the threshold of business self-service analysis; reduce the cost of repeated data R & D through data reuse and standardization construction.

[0271] According to an embodiment of the present disclosure, Figure 5 is the structural diagram of the data query processing device in the embodiments of the present disclosure. The embodiments of the present disclosure are applicable to the situation of receiving a query statement sent by any client accessing, performing corresponding queries on any database accessed, and receiving the query result and returning it to the client. The device is implemented by software and / or hardware and is specifically configured in an electronic device with certain data computing capabilities.

[0272] Such as Figure 5 shown, a data query processing device 500 includes: a query statement receiving module 501, a parameter configuration obtaining module 502, a standard statement determining module 503, a standard statement executing module 504, and a query result receiving module 505. Among them,

[0273] The query statement receiving module 501 is used to receive the query statement; the query statement is sent by calling the standard service interface;

[0274] The parameter configuration obtaining module 502 is used to detect the query parameters associated with the query statement and the standard configuration information of each query parameter;

[0275] A standard statement determination module 503, configured to determine a standard statement according to the standard configuration information of each of the query parameters and the query statement;

[0276] A standard statement execution module 504, configured to send the standard statement to the accessed database so that the database executes the standard statement;

[0277] A query result receiving module 505, configured to receive the query result fed back by the database and give feedback on the query statement.

[0278] Optionally, the parameter configuration acquisition module 502 includes:

[0279] A statement parsing unit, configured to parse the query statement to obtain at least one query object included in the query statement;

[0280] A parameter acquisition unit, configured to acquire object parameters of a standard data model;

[0281] A parameter determination unit, configured to, in response to determining that there are object parameters corresponding to the query object, determine at least one query parameter associated with the query statement according to the existing object parameters corresponding to the query object;

[0282] A configuration determination unit, configured to obtain the standard configuration information of each of the query parameters according to the standard data model.

[0283] Optionally, the standard data model includes: a data table model;

[0284] The data query processing device further includes: a data table model generation module, configured to:

[0285] Obtain the fields included in at least one data table in the accessed database;

[0286] Establish a mapping relationship between each of the data tables according to the fields included in each of the data tables;

[0287] Generate a data table model according to the mapping relationship between each of the data tables and the fields included in each of the data tables, and use it as the standard data model.

[0288] Optionally, the standard statement execution module 504 includes:

[0289] A query range determination unit, configured to obtain the query range corresponding to the standard statement;

[0290] A query range sending unit, configured to send the standard statement and the corresponding query range to the accessed database so that the database executes the standard statement within the corresponding query range.

[0291] Optionally, the query range determination unit includes:

[0292] A data table screening subunit, configured to screen, in each of the data tables, a data table corresponding to the standard statement according to each query field in the standard statement and the fields included in each of the data tables;

[0293] A range determination subunit, configured to determine the data table corresponding to the standard statement as the query range corresponding to the standard statement.

[0294] Optionally, the data query processing device further includes:

[0295] A traceability generation module, configured to generate traceability data of the query result according to the standard statement and the corresponding data table after determining the data table corresponding to the standard statement as the query range corresponding to the standard statement;

[0296] A traceability feedback module, configured to give feedback on the query statement according to the traceability data.

[0297] Optionally, the standard data model includes: a parameter model;

[0298] The data query processing device further includes: a data table model generation module, configured to:

[0299] Receive a parameter configuration request;

[0300] Generate object parameters according to the parameter configuration request, and standard configuration information corresponding to the generated object parameters;

[0301] Add the generated object parameters and the standard configuration information corresponding to the generated object parameters to the parameter model.

[0302] Optionally, the standard statement determination module 503 includes:

[0303] A statement rewriting detection unit, configured to determine whether to rewrite the query statement according to each query parameter to obtain a rewriting detection result;

[0304] A rewriting type determination unit, configured to detect the parameter types of each query parameter according to the standard configuration information of each query parameter in response to determining that the rewriting detection result indicates rewriting;

[0305] A standard statement generation unit, configured to rewrite the query statement according to each parameter type to obtain a standard statement;

[0306] A standard statement determination unit, configured to determine the query statement as the standard statement in response to determining that the rewriting detection result indicates keeping the original sentence.

[0307] Optionally, the rewriting type determination unit includes:

[0308] The non-aggregation type determination subunit is configured to determine that the parameter type is the replacement type in response to determining that the standard configuration information includes a non-aggregation calculation formula;

[0309] The low-aggregation type determination subunit is configured to determine that the parameter type is the nested type in response to determining that the standard configuration information includes a low-aggregation calculation formula;

[0310] The high-aggregation type determination subunit is configured to determine that the parameter type is the fusion type in response to determining that the standard configuration information includes a high-aggregation calculation formula.

[0311] Optionally, the standard statement generation unit includes:

[0312] The replacement data determination subunit is configured to determine the replacement data of each query parameter according to the standard configuration information in response to determining that the parameter type is the replacement type;

[0313] The statement content replacement subunit is configured to replace the corresponding content in the query statement with each replacement data to obtain the standard statement.

[0314] Optionally, the standard statement generation unit includes:

[0315] The nested statement determination subunit is configured to determine the statements of each query parameter according to the standard configuration information of each query parameter in response to determining that the parameter type is the nested type;

[0316] The statement content nesting subunit is configured to nest the statements of each query parameter into the query statement to obtain the standard statement.

[0317] Optionally, the standard statement generation unit includes:

[0318] The nested statement determination subunit is configured to determine the statements of each query parameter and the statements of the sub-parameters in each query parameter according to the standard configuration information of each query parameter in response to determining that the parameter type is the fusion type;

[0319] The sub-statement fusion subunit is configured to perform dimensional fusion on the statements of each sub-parameter in each query parameter to obtain the sub-statements of each query parameter;

[0320] The sub-statement nesting subunit is configured to nest the sub-statements of each query parameter into the statements of each query parameter to obtain the fusion statements of each query parameter;

[0321] A fusion statement nesting subunit for nesting the fusion statements of the query parameters into the statements of the query parameters;

[0322] A statement content nesting subunit for nesting the fusion statement into the query statement to obtain the standard statement.

[0323] Optionally, the standard statement generation unit includes:

[0324] A syntax tree acquisition subunit for acquiring the syntax tree of the query statement;

[0325] A syntax tree update subunit for updating the syntax tree of the query statement according to the parameter types of the query parameters and the standard configuration information;

[0326] A syntax tree conversion subunit for generating a standard statement according to the updated syntax tree.

[0327] Optionally, the query statement receiving module 501 includes:

[0328] A client interaction unit for receiving a query statement sent by a client, where the query statement is sent by calling a standard service interface configured in the client through a visualization application.

[0329] The above data query processing device can execute the data query processing method provided in any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the data query processing method.

[0330] In the technical solution of the present disclosure, the processing of collection, storage, use, processing, transmission, provision, and disclosure of the involved question-and-answer data all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0331] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0332] Figure 6 The schematic regional diagram of an example electronic device 600 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein. The electronic device 600 can be a data query processing device.

[0333] As shown Figure 6 in FIG. 684, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required by the electronic device 600 instructions can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0334] A plurality of components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disc, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0335] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the data query processing method. For example, in some embodiments, the data query processing method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the data query processing method described above can be executed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the data query processing method in any other suitable manner (e.g., by means of firmware).

[0336] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0337] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / instructions specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0338] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0339] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0340] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0341] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services. The server can also be a server of a distributed system, or a server combined with blockchain.

[0342] Artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and it has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, and knowledge graph technology.

[0343] Cloud computing refers to a technical system that accesses an elastic and scalable shared physical or virtual resource pool through a network. The resources can include servers, instruction systems, networks, software, applications, storage devices, etc., and the resources can be deployed and managed in an on-demand and self-service manner. Through cloud computing technology, it can provide efficient and powerful data processing capabilities for the application of technologies such as artificial intelligence and blockchain and model training.

[0344] It should be understood that various forms of processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions provided in this disclosure can be achieved. There is no limitation herein.

[0345] Figure 7 It is a schematic diagram of a data query system disclosed according to an embodiment of the present disclosure. As Figure 7 shown, the data query system 700 includes: a client 701, a data query processing device 702 according to any one of the embodiments of the present disclosure, and a database 703; the client 701 is communicatively connected to the data query processing device 702, and the data query processing device 702 is communicatively connected to the database 703;

[0346] The client 701 is used to send a query statement to the data query processing device 702;

[0347] The data query processing device 702 is used to generate a standard statement according to the query statement, send the standard statement to the database 703, and feedback the query result fed back by the database 703 for the standard statement to the client 701;

[0348] The database 703 is used to execute the standard statement, obtain the query result, and feedback the query result to the data query processing device 702.

[0349] Among them, the client can be any client that can access the database. The data structure of the database can be heterogeneous data. The data source of the database is not limited.

[0350] According to the technical solution of the present disclosure, the data query processing device provides a standard service interface for the client, and at the same time can provide a unified standard data query service for clients with different business requirements, and can interface with databases with different data structures and data sources. Thus, the data model and parameter system are unified from input to output, ensuring the consistency of data caliber, and can be flexibly adapted to different clients and databases, avoiding the construction cost of re-research and development for specific clients and specific databases, improving the reusability of data services, and reducing the cost of repeated construction.

[0351] Optionally, the client 701 includes: a business intelligence client.

[0352] Among them, BI tools are usually used in the data analysis field of enterprises, especially in scenarios that require cross-topic self-service analysis. The BI client can communicate and connect with the data query processing device based on Apache remote JDBC.

[0353] By providing a unified data model management system through the data query processing device, enterprises can achieve data standardization, reusability, and full-life cycle management, improve the efficiency and accuracy of data analysis, and provide flexible parameter definition methods and cross-topic analysis capabilities, reducing the threshold of business self-service analysis.

[0354] In some embodiments, the overall architecture of the data query system is as Figure 8 shown. Among them, the data query system includes a visualization application layer, a model service layer, and a physical parameter layer. Among them, the visualization application layer represents the client. The model service layer represents the data query processing device. The underlying IT data warehouse is a database storing physical tables.

[0355] Among them, the visualization application layer can be the layer where the client of the BI product is located. Specifically, enterprises can use the functions of the data query processing device based on the client of the mid-platform BI product. The mid-platform BI product meets some common business needs of enterprises. For example, it can generate and display BI reports, monitoring dashboards, and operation dashboards. Among them, the mid-platform can refer to the middle-layer architecture between the front end and the back end. In addition, enterprises can also conduct self-developed integration based on the mid-platform BI product and embed their own developed data products, and the self-developed data products can meet the specific business needs of the enterprises themselves. For example, the self-developed data products can generate and display human resource reports, administrative dashboards, and financial dashboards. In addition, the visualization application layer can also include third-party BI products.

[0356] The data query processing device mainly includes a model layer and a service layer. The service layer can access the clients of any BI products in the visualization application layer, such as local mid-platform BI products, self-developed data products based on the mid-platform BI product, and third-party BI products, etc. The model layer focuses on building a unified data model. The data model includes a data table model and a parameter model, decoupling data modeling from the BI product and becoming a reusable data service. The service layer provides systematic and online data asset management capabilities and service capabilities, supporting data standardization construction and management. The service layer provides external standard service interfaces for the clients in the visualization application layer, and the clients in the visualization application layer call various services in the service layer through the standard service interfaces. The service layer generates standard statements and calls the IT data warehouse. The IT data warehouse executes the standard statements, performs data query, aggregation, and summarization in the physical tables, and feeds back the query results. The service layer receives the feedback results and feeds them back to the client.

[0357] Among them, the service layer is a standard data service, and the standard data service may include a data asset service, a data query service, and a data security service. Among them, the data asset service and the data security service are background services. The data asset service can provide functions such as a data map, data assets, and data lineage. The data query service provides a data query processing service, that is, the data query processing method provided by the embodiments of the present disclosure. The data query service can provide functions such as SQL query, parameter interface, and metadata query. The data security service can provide functions such as permission configuration, application, authentication, and security audit. When the service layer receives a query statement from the client, it can authenticate the client based on the account, password, and database sent by the client at the same time. When the authentication passes, it starts to process the query statement; when the authentication fails, it can feedback a response result indicating no permission.

[0358] Among them, the standard data model in the model layer may include a parameter model, and the parameter model may include configured query parameters and standard configuration information of the query parameters. Among them, the configured query parameters may be derived parameters. The standard data model in the model layer may include a data table model, and the data table model may include metadata of the underlying physical table and mapping relationships of the physical table, etc.

[0359] The underlying IT data warehouse is used to store the real source data of the physical table, and the metadata of the physical table can be provided to the data query processing device to create a parameter model and a data table model.

[0360] The embodiments of the present disclosure can access the visualization application layer by using a general access method. For example, it can communicate with the visualization application layer based on a communication connection method driven by a general database, and this communication connection method is compatible with the access of most BI products on the market; through custom SQL syntax tree parsing and automated parameter derivation mechanism, it supports real-time parameter calculation and multi-dimensional aggregation; by adopting a separated design of the model layer (unified metadata management) and the service layer (API-based data service of application programming interface), and adopting a decoupled hierarchical architecture, it supports high-concurrency access and elastic expansion; the service layer can also realize full-linkage data lineage tracing: build a data lineage graph based on a graph database, realize field-level traceability and impact analysis, and ensure data consistency; and virtualize the physical table, build a data table model of the virtual table, map multi-source heterogeneous data to a unified semantic layer, and reduce the complexity of business queries.

[0361] The above specific implementation manners do not constitute a limitation to the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A data query processing method, comprising: Receiving a query statement; The query statement is sent by calling a standard service interface; Detecting query parameters associated with the query statement and standard configuration information of each of the query parameters; Determining a standard statement according to the standard configuration information of each of the query parameters and the query statement; Sending the standard statement to the accessed database so that the database executes the standard statement; Receiving a query result fed back by the database and providing feedback for the query statement.

2. The method according to claim 1, wherein, The detecting query parameters associated with the query statement and standard configuration information of each of the query parameters includes: Parsing the query statement to obtain at least one query object included in the query statement; Obtaining object parameters of a standard data model; In response to determining that there are object parameters corresponding to the query object, determining at least one query parameter associated with the query statement according to the existing object parameters corresponding to the query object; Obtaining the standard configuration information of each of the query parameters according to the standard data model.

3. The method according to claim 2, wherein The standard data model includes: a data table model; The data table model is generated by the following method: Obtaining fields included in at least one data table in the accessed database; Establishing a mapping relationship between each of the data tables according to the fields included in each of the data tables; Generating a data table model according to the mapping relationship between each of the data tables and the fields included in each of the data tables, and using it as the standard data model.

4. The method according to claim 3, wherein The sending the standard statement to the accessed database so that the database executes the standard statement includes: Obtaining a query range corresponding to the standard statement; Sending the standard statement and the corresponding query range to the accessed database so that the database executes the standard statement within the corresponding query range.

5. The method according to claim 4, wherein The obtaining a query range corresponding to the standard statement includes: Filtering data tables corresponding to the standard statement in each of the data tables according to each query field in the standard statement and the fields included in each of the data tables; Determining the data table corresponding to the standard statement as the query range corresponding to the standard statement.

6. The method according to claim 5, wherein, After determining the data table corresponding to the standard statement as the query range corresponding to the standard statement, it further includes: Generating traceability data of the query result according to the standard statement and the corresponding data table; Providing feedback for the query statement according to the traceability data.

7. The method according to claim 2, wherein The standard data model includes: a parameter model; The parameter model is generated by the following method: Receiving a parameter configuration request; Generating object parameters according to the parameter configuration request and standard configuration information corresponding to the generated object parameters; Adding the generated object parameters and the standard configuration information corresponding to the generated object parameters to the parameter model.

8. The method according to claim 1, wherein The determining a standard statement according to the standard configuration information of each of the query parameters and the query statement includes: Determining whether to rewrite the query statement according to each of the query parameters to obtain a rewrite detection result; In response to determining that the rewritten detection result indicates a rewrite, according to the standard configuration information of each of the query parameters, detect the parameter types of each of the query parameters; Rewrite the query statement according to each of the parameter types to obtain a standard statement; In response to determining that the rewritten detection result indicates keeping the original sentence, determine the query statement as the standard statement.

9. The method according to claim 8, wherein, The detecting the parameter types of each of the query parameters according to the standard configuration information of each of the query parameters includes: In response to determining that the standard configuration information includes a non-aggregation calculation formula, determine that the parameter type is a replacement type; In response to determining that the standard configuration information includes a low-aggregation calculation formula, determine that the parameter type is a nested type; In response to determining that the standard configuration information includes a high-aggregation calculation formula, determine that the parameter type is a fusion type.

10. The method according to claim 8, wherein, The rewriting the query statement according to each of the parameter types to obtain a standard statement includes: In response to determining that the parameter type is a replacement type, according to the standard configuration information, determine the replacement data of each of the query parameters; Replace the corresponding content in the query statement according to each of the replacement data to obtain the standard statement.

11. The method according to claim 8, wherein, The rewriting the query statement according to each of the parameter types to obtain a standard statement includes: In response to determining that the parameter type is a nested type, according to the standard configuration information of each of the query parameters, determine the statements of each of the query parameters; Nest the statements of each of the query parameters into the query statement to obtain the standard statement.

12. The method according to claim 8, wherein, The rewriting the query statement according to each of the parameter types to obtain a standard statement includes: In response to determining that the parameter type is a fusion type, according to the standard configuration information of each of the query parameters, determine the statements of each of the query parameters and the statements of the sub-parameters in the query parameters; Perform dimension fusion on the statements of each of the sub-parameters in each of the query parameters to obtain the sub-statements of each of the query parameters; Nest the sub-statements of each of the query parameters into the statements of each of the query parameters to obtain the fusion statements of each of the query parameters; Nest the fusion statements of each of the query parameters into the statements of the query parameters; Nest the fusion statement into the query statement to obtain the standard statement.

13. The method according to claim 8, wherein The rewriting the query statement according to each of the parameter types to obtain a standard statement includes: Obtain the syntax tree of the query statement; Update the syntax tree of the query statement according to the parameter types and standard configuration information of each of the query parameters; Generate a standard statement according to the updated syntax tree.

14. The method according to claim 1, wherein, The receiving the query statement includes: Receive the query statement sent by the client, and the query statement is sent by calling the standard service interface configured in the client.

15. A data query processing device, including: A query statement receiving module, configured to receive a query statement; The query statement is sent by calling a standard service interface; A parameter configuration obtaining module, configured to detect the query parameters associated with the query statement and the standard configuration information of each of the query parameters; A standard statement determination module, configured to determine a standard statement according to the standard configuration information of each of the query parameters and the query statement; A standard statement execution module, configured to send the standard statement to the accessed database, so that the database executes the standard statement; A query result receiving module, configured to receive the query result fed back by the database and give a feedback for the query statement.

16. A data query processing device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the data query processing method according to any one of claims 1-14.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the data query processing method according to any one of claims 1-14.

18. A computer program product, comprising a computer program which, when executed by a processor, implements the data query processing method according to any one of claims 1-14.

19. A data query system, comprising: A client, the data query processing device according to claim 16, and a database; The client is communicatively connected to the data query processing device, and the data query processing device is communicatively connected to the database; The client is configured to send a query statement to the data query processing device; The data query processing device is configured to generate a standard statement according to the query statement, send the standard statement to the database, and feed back the query result fed back by the database for the standard statement to the client; The database is configured to execute the standard statement to obtain the query result and feed back the query result to the data query processing device.

20. The method according to claim 19, wherein, The client includes: a business intelligence client.

Images