Data processing method and device, storage medium and computing equipment

Abstract

The embodiment of the invention provides a data processing method and device, a storage medium and computing equipment. The method is applied to an index platform corresponding to a data analysis system, and the index platform provides a self-defined database connection interface. The method comprises the following steps: receiving an index query statement sent by the data analysis system calling the self-defined database connection interface; wherein the index query statement comprises a target index defined in the index platform; determining an index model where the target index is located and constructing a target database of the index model; converting the index query statement into a target database query statement supported by the target database; and sending the target database query statement to the target database, querying target data corresponding to the target index in the target database, and returning the queried target data to the data analysis system.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more specifically, the embodiments of the present disclosure relate to a data processing method, apparatus, storage medium, and computing device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present disclosure. No description herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] In businesses involving data processing, data processing can usually be achieved with the help of an indicator platform.

[0004] In the indicator platform, users can define relevant indicators or indicator models (several related indicators can constitute an indicator model) according to business needs. These indicators describe how data is calculated. For example, a user can define an indicator as "group by region field and province field, and calculate the sum of sales field in each group."

[0005] After the indicator platform processes the business data based on the calculation method of the defined indicator description and obtains the data corresponding to the indicator, the data can be further visualized and displayed through the data analysis system. This can help enterprises, organizations or individuals track business details, operational efficiency, market performance and achievement of predetermined goals in real time or non-real time.

[0006] In actual applications, since there is no unified standard for indicator model definition or indicator definition in the indicator platform in the industry, the connection between the indicator platform and the data analysis system usually requires complex adaptation and synchronization.

[0007] Due to the lack of unified standards, each data analysis system needs to be adapted separately to convert the indicator definitions of the indicator platform into the fields of the data analysis system itself, which increases the complexity and time cost of adaptation. In addition, the adaptation process of different data analysis systems may be different, resulting in the need for customized development and maintenance for each data analysis system. Summary of the invention

[0008] In a first aspect of the embodiments of the present disclosure, a data processing method is provided. The method is applied to an indicator platform corresponding to a data analysis system, wherein the indicator platform provides a customized database connection interface, wherein the customized database connection interface includes an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the method includes:

[0009] Receiving an indicator query statement sent by the data analysis system calling the customized database connection interface; wherein the indicator query statement includes a target indicator defined in the indicator platform;

[0010] Determine the indicator model where the target indicator is located and construct a target database for the indicator model;

[0011] Converting the index query statement into a target database query statement supported by the target database;

[0012] The target database query statement is sent to the target database, the target database is queried for target data corresponding to the target indicator, and the queried target data is returned to the data analysis system.

[0013] Optionally, converting the indicator query statement into a target database query statement supported by the target database includes:

[0014] Obtaining a field value connected after the data table field in the indicator query statement, and determining the field value as the indicator name of the indicator platform;

[0015] According to the indicator calculation logic defined by the indicator platform for the indicator name, an indicator calculation statement corresponding to the indicator calculation logic is generated using a target database language supported by the target database;

[0016] The indicator query statement is converted into a target indicator query statement in the target database language, and the field value connected after the data table field in the target indicator query statement is replaced with the indicator calculation statement, so as to obtain a target database query statement.

[0017] Optionally, the querying the target database for target data corresponding to the target indicator includes:

[0018] According to the target field corresponding to the target indicator in the target database query statement, query the target field data in the target database;

[0019] According to the indicator calculation statement in the target database query statement, the indicator value corresponding to the target field data is calculated, and the indicator value is determined as the target data corresponding to the target indicator.

[0020] Optionally, the method further includes:

[0021] Receiving a metadata query request sent by the data analysis system by calling the custom database connection interface;

[0022] In response to the metadata query request, the indicator model constructed in the indicator platform is returned to the data analysis system; so that the data analysis system constructs an indicator query statement based on the indicators defined by the indicator model.

[0023] Optionally, the data analysis system includes a BI system, and the database connection interface includes a JDBC interface.

[0024] In a second aspect of the embodiments of the present disclosure, a data processing method is provided, which is applied to a data analysis system corresponding to an indicator platform, wherein the indicator platform provides a customized database connection interface, and the customized database connection interface is an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the method comprises:

[0025] Obtaining a generated indicator query statement for a target database; wherein the indicator query statement includes a target indicator defined in the indicator platform;

[0026] The database connection interface for connecting to the target database called by the indicator query statement is replaced with the custom database connection interface, and the indicator query statement is forwarded to the indicator platform through the custom database connection interface;

[0027] Receive target data returned by the indicator platform; the target data is indicator data corresponding to the target indicator queried from the target database after the indicator platform converts the indicator query statement into a target database query statement supported by the target database.

[0028] Optionally, the method further includes:

[0029] Calling the custom database connection interface to send a metadata query request to the indicator platform;

[0030] Receiving the indicator model returned by the indicator platform;

[0031] The obtained index query statement for the target database includes:

[0032] Obtain the target indicator defined based on the indicator model and construct an indicator query statement.

[0033] Optionally, the data analysis system includes a BI system, and the database connection interface includes a JDBC interface.

[0034] In a third aspect of the embodiments of the present disclosure, a data processing device is provided, which is applied to an indicator platform corresponding to a data analysis system, wherein the indicator platform provides a customized database connection interface, and the customized database connection interface includes an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the device includes:

[0035] A receiving unit receives an indicator query statement sent by the data analysis system calling the custom database connection interface; wherein the indicator query statement includes a target indicator defined in the indicator platform;

[0036] A determination unit, which determines the indicator model where the target indicator is located and constructs a target database for the indicator model;

[0037] A conversion unit, converting the indicator query statement into a target database query statement supported by the target database;

[0038] The processing unit sends the target database query statement to the target database, queries the target database for target data corresponding to the target indicator, and returns the queried target data to the data analysis system.

[0039] Optionally, the conversion unit includes:

[0040] An acquisition subunit is provided to acquire a field value connected after the data table field in the indicator query statement, and determine the field value as the indicator name of the indicator platform;

[0041] A generating subunit generates an indicator calculation statement corresponding to the indicator calculation logic using a target database language supported by the target database according to the indicator calculation logic defined by the indicator platform for the indicator name;

[0042] The conversion subunit converts the indicator query statement into a target indicator query statement in the target database language, and replaces the field value connected after the data table field in the target indicator query statement with the indicator calculation statement, thereby obtaining a target database query statement.

[0043] Optionally, when querying the target data corresponding to the target indicator in the target database, the processing unit is used to query the target field data in the target database according to the target field corresponding to the target indicator in the target database query statement; and further calculate the indicator value corresponding to the target field data according to the indicator calculation statement in the target database query statement, and determine the indicator value as the target data corresponding to the target indicator.

[0044] Optionally, the receiving unit is further used to receive a metadata query request sent by the data analysis system calling the custom database connection interface;

[0045] The processing unit is also used to return the indicator model constructed in the indicator platform to the data analysis system in response to the metadata query request; so that the data analysis system constructs an indicator query statement based on the indicators defined by the indicator model.

[0046] Optionally, the data analysis system includes a BI system, and the database connection interface includes a JDBC interface.

[0047] In a fourth aspect of the embodiments of the present disclosure, a data processing device is provided, which is applied to a data analysis system corresponding to an indicator platform, wherein the indicator platform provides a customized database connection interface, and the customized database connection interface is an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the device comprises:

[0048] An acquisition unit is configured to acquire a generated indicator query statement for a target database; wherein the indicator query statement includes a target indicator defined in the indicator platform;

[0049] A replacement unit replaces the database connection interface called by the indicator query statement for connecting to the target database with the custom database connection interface, and forwards the indicator query statement to the indicator platform through the custom database connection interface;

[0050] A receiving unit receives target data returned by the indicator platform; the target data is indicator data corresponding to the target indicator queried from the target database after the indicator platform converts the indicator query statement into a target database query statement supported by the target database.

[0051] Optionally, the device further comprises:

[0052] A query unit, calling the custom database connection interface to send a metadata query request to the indicator platform, and receiving an indicator model returned by the indicator platform;

[0053] The acquisition unit is further used to acquire an indicator query statement constructed based on the target indicator defined by the indicator model.

[0054] Optionally, the data analysis system includes a BI system, and the database connection interface includes a JDBC interface.

[0055] In a fifth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, comprising:

[0056] When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data processing method as described in any of the preceding items.

[0057] In a sixth aspect of the embodiments of the present disclosure, there is provided a computing device, including:

[0058] processor;

[0059] a memory for storing instructions executable by the processor;

[0060] Wherein, the processor is configured to execute the executable instructions to implement the data processing method as described in any of the preceding items.

[0061] According to the data processing solution provided in the embodiment of the present disclosure, by logically decoupling the data analysis system from the indicator platform, the data analysis system no longer needs to incur additional customized development costs to adapt to the indicator platform.

[0062] Specifically, the indicator platform is regarded as a special database, and the indicator platform implements a custom database connection interface for connecting to its own. In this way, when the data analysis system sends an indicator query statement to the indicator platform through the custom database connection interface, since the indicator query statement is for the indicator of the indicator platform rather than the data table in the actual database, the indicator platform can convert the indicator query statement into a target database query statement that can be used by the actual database, thereby querying the actual database to obtain the indicator data. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0064] Figure 1 The system diagram of the data processing system provided by the present disclosure is schematically shown;

[0065] Figure 2 The data processing method provided by the present disclosure is schematically shown;

[0066] Figure 3 Schematically shows another data processing method provided by the present disclosure;

[0067] Figure 4 The following schematically shows a data processing method for multi-party interaction provided by the present disclosure;

[0068] Figure 5The medium schematic diagram provided by the present disclosure is schematically shown;

[0069] Figure 6 The data processing device provided by the present disclosure is schematically shown;

[0070] Figure 7 The data processing device provided by the present disclosure is schematically shown;

[0071] Figure 8 The schematic diagram of the computing device provided by the present disclosure is schematically shown.

[0072] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0073] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0074] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0075] According to an embodiment of the present disclosure, a data processing method, a computer-readable storage medium, an apparatus, and a computing device are proposed.

[0076] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0077] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0078] The data involved in the present disclosure may be data authorized by the user or fully authorized by all parties. The collection, dissemination, and use of the data shall comply with the requirements of relevant national laws and regulations. The implementation methods / examples of the present disclosure may be combined with each other. SUMMARY OF THE INVENTION

[0080] The present disclosure aims to provide a data processing solution, which decouples the logic of the data analysis system from the indicator platform, so that the data analysis system no longer needs to spend additional customized development costs to adapt to the indicator platform. Specifically, the indicator platform is regarded as a special database, and the indicator platform implements a customized database connection interface for connecting to itself, so that when the data analysis system sends an indicator query statement to the indicator platform through the customized database connection interface, since the indicator query statement is aimed at the indicator of the indicator platform, rather than the data table in the actual database, the indicator query statement can be converted by the indicator platform into a target database query statement that can be used by the actual database, so as to query the actual database to obtain the indicator data.

[0081] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0082] Application Scenario Overview

[0083] First reference Figure 1 A system architecture diagram of a data processing system is shown in FIG. In the system architecture diagram, the data processing system may include a data analysis system, an indicator platform, and a database.

[0084] The data analysis system may include but is not limited to a BI (Business Intelligence) system, a data mining system, a data prediction system, and the like.

[0085] The BI system may refer to a collection of tools and technologies used to collect, analyze, and present business data. The BI system can help enterprises transform raw data into valuable information to support the decision-making process. BI systems can generally include data warehouses, data mining, analysis tools, and visualization capabilities. By implementing a BI system, enterprises can improve their ability to make data-driven decisions, thereby better responding to competition and market changes.

[0086] The indicator platform can be a platform that provides standardized definitions and management of indicators. The indicator platform can provide a single, reliable data source for business decision-making and management, solving the long-standing problem of inconsistent indicator calibers.

[0087] The database stores the raw data required by the indicator platform to calculate indicators. The database may include various mainstream or non-mainstream databases, such as NySQL, Oracle, Hive, GreenPlum, etc.

[0088] When in use, the indicator platform can use the database connection interface provided by the database to initiate relevant data query requests to the database, calculate corresponding indicators based on the data queried from the database, and further send the calculated indicators to the data analysis system.

[0089] In this specification, the indicator platform regards itself as a special database and constructs a custom database connection interface for connecting to itself; in this way, the data analysis system can initiate a data query request to the indicator platform through the custom database connection interface.

[0090] Different types of database connection interfaces may be used for different database connection technologies. For example, for JAVA technology, the database connection interface may refer to a JDBC (Java Database Connectivity) interface, which is an API for database connection in Java. Through the JDBC interface, a standard database language (such as SQL) may be used to interact with the database. For another example, for Python technology, the database connection interface may refer to a DB-API interface.

[0091] In addition, the database connection interface may also include but is not limited to ODBC interface, ADO.NET interface, etc. The use of various database connection interfaces can be flexibly adjusted according to business needs, and this embodiment of the specification does not limit this.

[0092] Exemplary Methods

[0093] Combine the following Figure 1 The application scenario shown is shown in Figure 2 To describe the method of data processing according to the exemplary embodiment of the present disclosure. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0094] The data processing method can be applied to the aforementioned indicator platform corresponding to the data analysis system, and the indicator platform provides a customized database connection interface, and the customized database connection interface includes an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform.

[0095] like Figure 2 As shown, the method may include the following steps:

[0096] Step 210, receiving an indicator query statement sent by the data analysis system calling the custom database connection interface; wherein the indicator query statement includes a target indicator defined in the indicator platform;

[0097] Step 220, determining the indicator model where the target indicator is located and constructing a target database for the indicator model;

[0098] Step 230, converting the indicator query statement into a target database query statement supported by the target database;

[0099] Step 240: Send the target database query statement to the target database, query the target database for target data corresponding to the target indicator, and return the queried target data to the data analysis system.

[0100] In the embodiments of this specification, the indicator platform provides a customized database connection interface for the data analysis system, and can also provide the data analysis system with various indicator information defined in the indicator platform, such as the name, type, meaning, function, etc. of the indicator.

[0101] For example, in addition to actively sending indicator information to the data analysis system, the data analysis system can also actively obtain indicator information from the indicator platform.

[0102] The data analysis system can regard the indicator platform as a special database, and send an indicator query statement to the indicator platform through a customized database connection interface provided by the indicator platform. The indicator query statement contains the target indicator that the data analysis system wants to query.

[0103] The customized database connection interface can be improved based on the original data connection interface of any type of destination database, and the improved database connection interface is the customized database connection interface. Specifically, the improvement is to transform the forwarding logic of the original database connection interface, and change the original forwarding logic (forwarding the database query statement to be sent to the destination database) to the new forwarding logic (forwarding the database query statement to be sent to the indicator platform). In this way, the data analysis system can connect and query the indicator data in the indicator platform through the customized connection interface just like connecting to an ordinary database.

[0104] The following takes GreenPlum as an example of an exemplary distributed data warehouse. GreenPlum provides a GreenPlum connection interface (generally called GreenPlum JDBC) to the outside world.

[0105] If the data analysis system directly calls the original GreenPlum connection interface to send an indicator query statement, the indicator query statement will be directly sent to GreenPlum.

[0106] Therefore, by modifying the forwarding logic of the GreenPlum connection interface, the forwarding destination address is changed from GreenPlum to the indicator platform; in this way, the data analysis system does not need to be specially adapted to the indicator platform, but directly uses the modified GreenPlum connection interface (that is, the customized connection interface) provided by the indicator platform to connect to the indicator platform disguised as GreenPlum, and query the indicator data to be queried.

[0107] In the indicator platform, one or more different databases can be connected, and one or more indicator models can be defined according to the data stored in the database or the business requirements related to these data. Each indicator model can contain one or more indicators. Among them, the indicator model has a corresponding relationship with the corresponding database.

[0108] In this way, the "schema-table-column" data structure used in the data analysis system can be matched with the "model-indicator-field" data structure of the indicator platform. When the user obtains the "schema" list in the data analysis system, the "model" list in the indicator platform is obtained; when the "table" list under a certain "schema" is obtained, the "indicator" list under the corresponding "model" in the indicator platform is obtained; all the "columns" obtained when querying a single "table" are all the "fields" in the "indicator" definition in the indicator platform. In this way, users can directly analyze the indicators defined in the indicator platform according to the normal use of the data analysis system, so the indicator platform does not need to synchronize the model and connection with the data analysis system.

[0109] As mentioned above, the data analysis system can actively obtain the defined indicator information from the indicator platform. Figure 1 As shown, in addition to sending indicator query statements, the data analysis system can also send metadata query requests; the metadata query request is used to synchronize the latest indicator model defined in the indicator platform.

[0110] Similar to the indicator query statement, the data analysis system can also send a metadata query request by calling a custom database connection interface. The custom database interface can forward the indicator query statement or metadata query request to the indicator platform.

[0111] Accordingly, the indicator platform may receive the metadata query request sent by the data analysis system calling the custom database connection interface;

[0112] After receiving the metadata query request, Figure 1 The business module shown in responds to the metadata query request and returns the indicator model constructed in the indicator platform to the data analysis system. In this way, the data analysis system constructs an indicator query statement based on the indicators defined in the indicator model.

[0113] After the indicator platform receives the indicator query statement sent by the data analysis system, it can further determine the indicator model where the target indicator contained in the indicator query statement is located, and the target database corresponding to the indicator model.

[0114] For example, database A stores the sales data of a certain product in various provinces. If the business needs to query the sales volume in different regions and provinces, then a sales model can be defined in the indicator platform, and an indicator named "Indicator A" can be further defined under the sales model. The definition of this indicator is to group by the "Region" and "Province" fields, and calculate the aggregate result of the sum of the field values ​​of the "Sales Volume" field.

[0115] Assuming that the data analysis department hopes to query the sum of sales of each province under the "region" of "Northeast", then the indicator query statement sent can contain "Indicator A", and the field value of the "Region" field in "Indicator A" is "Northeast", and the "Province" field is "Heilongjiang, Jilin and Liaoning" (or "All").

[0116] The indicator platform can determine the indicator model described by "Indicator A" according to "Indicator A" in the indicator query statement, thereby obtaining that the target database corresponding to the indicator model is database A.

[0117] Since the data is actually stored in the target database, the indicator platform needs to further query the target database for the data used to calculate indicator A. The forwarding logic of the indicator query statement has been modified, and the query is for "indicator A". Therefore, the target database cannot be queried using the indicator query statement, and the "indicator A" field does not exist in the target database. Therefore, the indicator platform also needs to convert the indicator query statement into a target database query statement that can connect to the target database and query data in the target database.

[0118] like Figure 1 As shown, the indicator platform can convert the indicator query statement into a target database query statement according to the data query and conversion module, that is, the aforementioned step 230.

[0119] In an exemplary embodiment, the aforementioned step 230 of converting the indicator query statement into a target database query statement supported by the target database may include:

[0120] Obtaining a field value connected after the data table field in the indicator query statement, and determining the field value as the indicator name of the indicator platform;

[0121] According to the indicator calculation logic defined by the indicator platform for the indicator name, an indicator calculation statement corresponding to the indicator calculation logic is generated using a target database language supported by the target database;

[0122] The indicator query statement is converted into a target indicator query statement in the target database language, and the field value connected after the data table field in the target indicator query statement is replaced with the indicator calculation statement, so as to obtain a target database query statement.

[0123] Let's take the indicator A mentioned above, where the data analysis system wants to query the sum of sales in each province under the "region" of "Northeast", as an example. Assume that database A that stores the sales data of a certain product in each province is a MySQL database, and these sales data are stored in the data table Table1 in the MySQL database; and the blueprint used by the custom database connection interface provided by the indicator platform is the GreenPlum connection interface. And further assume that the indicator query statement sent by the data analysis system can be as follows (using GreenPlum syntax):

[0124] SELECT "province", SUM ("sales")

[0125] FROM "Indicator A"

[0126] WHERE "Region" = 'Northeast'

[0127] GROUP BY "province"

[0128] It is worth noting that "Indicator A" in the indicator query statement is the indicator name in the indicator platform, not the actual data table in database A, and database A is a MySQL database instead of a GreenPlum data warehouse. There are differences in the query syntax between MySQL and GreenPlum.

[0129] Therefore, the indicator query statement sent by the data analysis system cannot be used directly to query database A. It is necessary to convert the indicator query statement into a target database query statement that meets the requirements of database A without changing the original query logic of the indicator query statement.

[0130] The process of converting an indicator query into a target database query statement can be divided into three steps: indicator extraction, indicator calculation statement generation, and query statement rewriting. These three steps are introduced in turn below.

[0131] 1) Indicator extraction: obtaining the field value connected after the data table field in the indicator query statement, and determining the field value as the indicator name of the indicator platform.

[0132] In a regular database query statement, a data table field can refer to the "FROM" field, and the field value connected after the "FROM" field is the data table name; and looking back at the indicator query statement in the above example, it can be found that the field value connected after the "FROM" field is the indicator name, that is, the data analysis system uses the indicator name as the data table name queried in the indicator query statement based on the aforementioned correspondence between "architecture-table-column" and "model-indicator-field".

[0133] Therefore, by extracting the field value connected after the data table field in the indicator query statement, it is possible to know which indicator defined by the indicator platform the data analysis system wants to query.

[0134] 2) Indicator calculation statement generation: according to the indicator calculation logic defined by the indicator platform for the indicator name, an indicator calculation statement corresponding to the indicator calculation logic is generated using the target database language supported by the target database.

[0135] After obtaining the indicator name of the target indicator, generate an indicator calculation statement with the same calculation logic that meets the requirements of the target database according to the indicator calculation logic of the target indicator defined in the indicator platform. This indicator calculation statement can be used to replace the field value connected after the data table field in the indicator query statement.

[0136] Let's take the indicator A mentioned above as an example, where the data analysis system wants to query the sum of sales of each province under the "region" of "Northeast". The definition of "Indicator A" in the indicator platform is to group by the two fields "region" and "province" under the MySQL table Table1, and then calculate the aggregate result of the sum (SUM) of the "sales" field; then the indicator calculation statement with the same calculation logic generated using the MySQL language can be as follows (using MySQL syntax):

[0137] SELECT `region`, `province`, SUM(`sales`)

[0138] ``

[0139] FROM Table1

[0140] GROUP BY `region`, `province`

[0141] 3) Query statement rewriting: converting the indicator query statement into a target indicator query statement in the target database language, and replacing the field value connected after the data table field in the target indicator query statement with the indicator calculation statement, thereby obtaining a target database query statement.

[0142] In this step, the indicator query statement originally written in GreenPlum syntax is rewritten into a target database query statement using MySQL syntax, and the target database query statement can be used to query data in the target database, i.e., MySQL database A. The specific rewriting content includes rewriting the indicator query statement using MySQL syntax and replacing the data table field, i.e., the field value after "FROM", with the indicator calculation statement.

[0143] Still taking the indicator A in the above data analysis system that wants to query the sum of sales of each province under the "region" of "Northeast" as an example, the target database query statement obtained by conversion (using MySQL syntax) is as follows:

[0144] SELECT `province`,SUM(`sales`)

[0145] FROM(

[0146] SELECT `region`, `province`, SUM(`sales`)

[0147] ``

[0148] FROM Table1

[0149] GROUP BY `region`, `province`

[0150] )AS`Indicator A`

[0151] WHERE `region` = 'Northeast'

[0152] GROUP BY province`

[0153] Since the converted target database query statement is written in the query language supported by the target database, and the data table to be queried (such as `Table1`) and the fields in the data table (such as `Region`, `Province`, `Sales`) actually exist in the target database, the target database can execute the target database query statement and return the queried data to the indicator platform, that is, the above-mentioned step 240.

[0154] In an exemplary embodiment, the above step 240 of querying the target database for target data corresponding to the target indicator may include:

[0155] According to the target field corresponding to the target indicator in the target database query statement, query the target field data in the target database;

[0156] According to the indicator calculation statement in the target database query statement, the indicator value corresponding to the target field data is calculated, and the indicator value is determined as the target data corresponding to the target indicator.

[0157] The following takes the target database query statement given in the previous example as an example. The target fields corresponding to the target indicator are `Region = Northeast`, `Province`, `Sales` and `Table1`, and then the sales of each province in Northeast China are queried from the `Table1` data table; then, according to the indicator calculation statement, the sum of the sales of each province queried is calculated, and the sum of the sales is the indicator data of indicator A.

[0158] After querying the indicator data of indicator A, the indicator platform can feed the indicator data back to the data analysis system, and the data processing process for the indicator query ends here.

[0159] After introducing the embodiment with the indicator platform as the execution body, the following further introduces the embodiment of the data processing method with the data analysis system corresponding to the indicator platform as the execution body. Similar to the above embodiment, the indicator platform provides a customized database connection interface, which is an interface that modifies the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform. Figure 3 As shown, the method embodiment with the data analysis system as the execution subject may include the following steps:

[0160] Step 310, obtaining a generated indicator query statement for a target database; wherein the indicator query statement includes a target indicator defined in the indicator platform.

[0161] The indicator query statements, customized database connection interfaces, etc. involved in this embodiment can refer to the previous embodiments and will not be described in detail here.

[0162] In an exemplary embodiment, the method further comprises:

[0163] Calling the custom database connection interface to send a metadata query request to the indicator platform;

[0164] Receiving the indicator model returned by the indicator platform;

[0165] The obtained index query statement for the target database includes:

[0166] Obtain the target indicator defined based on the indicator model and construct an indicator query statement.

[0167] In this example, the data analysis system can request the indicator platform to synchronize the latest indicator model through the custom database connection interface provided by the indicator platform, so as to construct an indicator query statement according to the target indicator defined in the latest indicator model.

[0168] The indicator query statement uses a database syntax that is consistent with the custom database connection interface.

[0169] Step 320: replace the database connection interface called by the indicator query statement for connecting to the target database with the custom database connection interface, and forward the indicator query statement to the indicator platform through the custom database connection interface.

[0170] The data analysis system regards the indicator platform as a special database, and initiates a request to query the indicator data corresponding to the target indicator to the indicator platform through a custom database connection interface.

[0171] Specifically, the data analysis system needs to replace the database connection interface originally used to connect to the target database with a customized database connection interface, so as to directly send indicator query statements to the indicator platform.

[0172] Step 330, receiving the target data returned by the indicator platform; the target data is the indicator data corresponding to the target indicator queried from the target database after the indicator platform converts the indicator query statement into a target database query statement supported by the target database.

[0173] After receiving the indicator query statement, the indicator platform can Figure 2 The embodiment shown performs relevant data processing and obtains target data corresponding to the target indicator, and then returns the target data to the data analysis system. The relevant data processing content performed by the indicator platform can be referred to the above embodiment, and will not be repeated here.

[0174] In the embodiments of this specification, the data analysis system is logically decoupled from the indicator platform, so that the data analysis system no longer needs to spend additional customized development costs to adapt to the indicator platform. Specifically, the indicator platform is regarded as a special database, and the indicator platform implements a customized database connection interface for connecting to itself. In this way, the data analysis system does not need to be specially adapted to the indicator platform. It can connect to the indicator platform disguised as a database through a customized connection interface just like connecting to an ordinary database, and query the indicator data to be queried. The indicator platform obtains the indicator data by querying the actual database by converting the indicator query statement into a target database query statement that can be used by the actual database.

[0175] After introducing the method embodiments with the indicator platform and the data analysis system as the execution subjects, the following introduces the embodiment of the performance analysis method under multi-party interaction in combination with the data analysis system, the indicator platform and the target database. Figure 4 As shown, it may include:

[0176] Step 410: The data analysis system obtains a generated indicator query statement for the target database; wherein the indicator query statement includes the target indicator defined in the indicator platform.

[0177] The indicator query statement uses a database syntax that is consistent with the custom database connection.

[0178] Step 412: The data analysis system replaces the database connection interface used to connect to the target database and called by the indicator query statement with the customized database connection interface.

[0179] The customized database connection interface can be improved based on the original data connection interface of any type of destination database, and the improved database connection interface is the customized database connection interface. Specifically, the improvement is to transform the forwarding logic of the original database connection interface, and change the original forwarding logic (forwarding the database query statement to be sent to the destination database) to the new forwarding logic (forwarding the database query statement to be sent to the indicator platform). In this way, the data analysis system can connect and query the indicator data in the indicator platform through the customized connection interface just like connecting to an ordinary database.

[0180] Step 414: The data analysis system forwards the indicator query statement to the indicator platform through the customized database connection interface.

[0181] The data analysis system regards the indicator platform as a special database, and initiates a request to query the indicator data corresponding to the target indicator to the indicator platform through a custom database connection interface. Specifically, the data analysis system needs to replace the database connection interface originally used to connect to the target database with a custom database connection interface, so as to directly send the indicator query statement to the indicator platform.

[0182] Step 420: the indicator platform receives the indicator query statement sent by the data analysis system by calling the custom database connection interface.

[0183] Step 422, the indicator platform determines the indicator model where the target indicator is located and constructs a target database for the indicator model.

[0184] In the indicator platform, one or more different databases can be connected, and one or more indicator models can be defined according to the data stored in the database or the business requirements related to these data. Each indicator model can contain one or more indicators. Among them, the indicator model has a corresponding relationship with the corresponding database.

[0185] After the indicator platform receives the indicator query statement sent by the data analysis system, it can further determine the indicator model where the target indicator contained in the indicator query statement is located, and the target database corresponding to the indicator model.

[0186] Step 424: the indicator platform converts the indicator query statement into a target database query statement supported by the target database.

[0187] Since the data is actually stored in the target database, the indicator platform needs to further query the target database for data used to calculate the target indicator. However, the forwarding logic of the indicator query statement has been modified, and the target indicator to be queried is not a field in the target database. Therefore, the target database cannot be queried using the indicator query statement.

[0188] To this end, the indicator platform also needs to convert the indicator query statement into a target database query statement that can connect to the target database and perform data query in the target database. For converting the indicator query statement to be able to connect to the target database, please refer to the above embodiment, which will not be repeated here.

[0189] Step 426: The indicator platform sends the target database query statement to the target database.

[0190] Step 430: the target database queries the target data corresponding to the target indicator according to the target database query statement.

[0191] Since the converted target database query statement is written in the query language supported by the target database, and the data table to be queried and the fields in the data table actually exist in the target database, the target database can execute the target database query statement and return the queried data to the indicator platform.

[0192] Step 432, the target database returns the queried target data to the indicator platform;

[0193] In step 440, the indicator platform further returns the target data returned by the target database to the data analysis system.

[0194] Step 450: The data analysis system receives the target data returned by the indicator platform.

[0195] After receiving the target data returned by the indicator platform, the data analysis system can display the target data, or further use the target data for other data processing, such as data analysis, data mining, data prediction, etc.

[0196] Exemplary Media

[0197] After introducing the method of the exemplary embodiment of the present disclosure, next, refer to Figure 5 A medium according to an exemplary embodiment of the present disclosure is described.

[0198] In this exemplary embodiment, the above method can be implemented by a program product, such as a portable compact disk read-only memory (CD-ROM) and including program code, and can be run on a device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus, or a device.

[0199] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0200] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0201] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RE, etc., or any suitable combination of the foregoing.

[0202] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0203] In summary, the present disclosure may provide a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device may be enabled to execute the aforementioned data processing method embodiment.

[0204] Exemplary Devices

[0205] After introducing the medium of the exemplary embodiment of the present disclosure, next, reference is made to Figure 6 and Figure 7 A device according to an exemplary embodiment of the present disclosure is described.

[0206] Figure 6 A block diagram of a data processing device according to an embodiment of the present disclosure is schematically shown, corresponding to the aforementioned Figure 2 The method embodiment shown is applied to an indicator platform corresponding to a data analysis system, the indicator platform provides a customized database connection interface, the customized database connection interface includes an interface for modifying the database query logic inside the database connection interface provided by any database to the indicator query logic of the indicator platform, and the data processing device may include:

[0207] The receiving unit 610 receives the indicator query statement sent by the data analysis system calling the customized database connection interface; wherein the indicator query statement includes the target indicator defined in the indicator platform;

[0208] A determination unit 620 is configured to determine an indicator model in which the target indicator is located and to construct a target database for the indicator model;

[0209] A conversion unit 630 converts the indicator query statement into a target database query statement supported by the target database;

[0210] The processing unit 640 sends the target database query statement to the target database, queries the target database for target data corresponding to the target indicator, and returns the queried target data to the data analysis system.

[0211] In an exemplary embodiment, the conversion unit 630 includes:

[0212] An acquisition subunit 631 acquires a field value connected after the data table field in the indicator query statement, and determines the field value as the indicator name of the indicator platform;

[0213] A generating subunit 633 generates an indicator calculation statement corresponding to the indicator calculation logic using a target database language supported by the target database according to the indicator calculation logic defined by the indicator platform for the indicator name;

[0214] The conversion subunit 635 converts the indicator query statement into a target indicator query statement in the target database language, and replaces the field value connected after the data table field in the target indicator query statement with the indicator calculation statement, thereby obtaining a target database query statement.

[0215] In an exemplary embodiment, when querying the target data corresponding to the target indicator in the target database, the processing unit 640 is used to query the target field data in the target database according to the target field corresponding to the target indicator in the target database query statement; and further calculate the indicator value corresponding to the target field data according to the indicator calculation statement in the target database query statement, and determine the indicator value as the target data corresponding to the target indicator.

[0216] In an exemplary embodiment, the receiving unit 610 is further configured to receive a metadata query request sent by the data analysis system by calling the custom database connection interface;

[0217] The processing unit 640 is further configured to return the indicator model constructed in the indicator platform to the data analysis system in response to the metadata query request, so that the data analysis system constructs an indicator query statement based on the indicators defined in the indicator model.

[0218] In an exemplary embodiment, the data analysis system includes a BI system, and the database connection interface includes a JDBC interface.

[0219] Figure 7 A block diagram of a data processing device according to an embodiment of the present disclosure is schematically shown, corresponding to the aforementioned Figure 3 The method embodiment shown is applied to a data analysis system corresponding to an indicator platform, wherein the indicator platform provides a customized database connection interface, wherein the customized database connection interface is an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform, and the data processing device may include:

[0220] The acquisition unit 710 acquires the generated indicator query statement for the target database; wherein the indicator query statement includes the target indicator defined in the indicator platform;

[0221] A replacement unit 720 replaces the database connection interface used to connect to the target database called by the indicator query statement with the custom database connection interface, and forwards the indicator query statement to the indicator platform through the custom database connection interface;

[0222] The receiving unit 730 receives the target data returned by the indicator platform; the target data is the indicator data corresponding to the target indicator queried from the target database after the indicator platform converts the indicator query statement into a target database query statement supported by the target database.

[0223] In an exemplary embodiment, the apparatus further comprises:

[0224] The query unit 701 calls the user-defined database connection interface to send a metadata query request to the indicator platform, and receives the indicator model returned by the indicator platform;

[0225] The acquisition unit 710 is further used to acquire an indicator query statement constructed based on the target indicator defined by the indicator model.

[0226] In an exemplary embodiment, the data analysis system includes a BI system, and the database connection interface includes a JDBC interface.

[0227] Exemplary Computing Devices

[0228] After introducing the method, medium and apparatus of the exemplary embodiments of the present disclosure, next, reference is made to Figure 8 A computing device according to an exemplary embodiment of the present disclosure is described.

[0229] Figure 8 The computing device 1500 shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0230] like Figure 8 As shown, computing device 1500 is in the form of a general computing device. Components of computing device 1500 may include, but are not limited to: at least one processing unit 1501, at least one storage unit 1502, and a bus 1503 connecting different system components (including processing unit 1501 and storage unit 1502).

[0231] The bus 1503 includes a data bus, a control bus, and an address bus.

[0232] The storage unit 1502 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 15021 and / or a cache memory 15022 , and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 15023 .

[0233] The storage unit 1502 may also include a program / utility 15025 having a set (at least one) of program modules 15024, such program modules 15024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0234] The computing device 1500 may also communicate with one or more external devices 1504 (eg, a keyboard, pointing device, etc.).

[0235] Such communication may be performed via input / output (I / O) interface 1505. In addition, computing device 1500 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via network adapter 1506. Figure 8 As shown, network adapter 1506 communicates with other modules of computing device 1500 via bus 1503. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with computing device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0236] Through Figure 8 The computing device 1500 shown can implement the aforementioned data processing method. More specifically, the storage unit 1502 stores instructions executable by the processing unit 1501. When the processing unit 1501 executes the instructions, the aforementioned data processing method is implemented.

[0237] It should be noted that, although several units / modules or sub-units / modules of the data processing device are mentioned in the above detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided into multiple units / modules to be embodied.

[0238] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0239] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A data processing method, applied to an indicator platform corresponding to a data analysis system, wherein the indicator platform provides a customized database connection interface, wherein the customized database connection interface includes an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the method comprises: Receiving an indicator query statement sent by the data analysis system calling the customized database connection interface; wherein the indicator query statement includes a target indicator defined in the indicator platform; Determine the indicator model where the target indicator is located and construct a target database for the indicator model; Converting the index query statement into a target database query statement supported by the target database; The target database query statement is sent to the target database, the target database is queried for target data corresponding to the target indicator, and the queried target data is returned to the data analysis system.

2. According to the method of claim 1, converting the indicator query statement into a target database query statement supported by the target database comprises: Obtaining a field value connected after the data table field in the indicator query statement, and determining the field value as the indicator name of the indicator platform; According to the indicator calculation logic defined by the indicator platform for the indicator name, an indicator calculation statement corresponding to the indicator calculation logic is generated using a target database language supported by the target database; The indicator query statement is converted into a target indicator query statement in the target database language, and the field value connected after the data table field in the target indicator query statement is replaced with the indicator calculation statement, so as to obtain a target database query statement.

3. According to the method of claim 2, the querying of the target data corresponding to the target indicator in the target database comprises: According to the target field corresponding to the target indicator in the target database query statement, query the target field data in the target database; According to the indicator calculation statement in the target database query statement, the indicator value corresponding to the target field data is calculated, and the indicator value is determined as the target data corresponding to the target indicator.

4. The method according to claim 1, further comprising: Receiving a metadata query request sent by the data analysis system by calling the custom database connection interface; In response to the metadata query request, the indicator model constructed in the indicator platform is returned to the data analysis system; so that the data analysis system constructs an indicator query statement based on the indicators defined by the indicator model. 5 . The method according to claim 1 , wherein the data analysis system comprises a BI system, and the database connection interface comprises a JDBC interface.

6. A data processing method, applied to a data analysis system corresponding to an indicator platform, wherein the indicator platform provides a customized database connection interface, wherein the customized database connection interface is an interface that modifies the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the method comprises: Obtaining a generated indicator query statement for a target database; wherein the indicator query statement includes a target indicator defined in the indicator platform; The database connection interface for connecting to the target database called by the indicator query statement is replaced with the custom database connection interface, and the indicator query statement is forwarded to the indicator platform through the custom database connection interface; Receive target data returned by the indicator platform; the target data is indicator data corresponding to the target indicator queried from the target database after the indicator platform converts the indicator query statement into a target database query statement supported by the target database.

7. A data processing device, applied to an indicator platform corresponding to a data analysis system, wherein the indicator platform provides a customized database connection interface, wherein the customized database connection interface includes an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the device comprises: A receiving unit receives an indicator query statement sent by the data analysis system calling the custom database connection interface; wherein the indicator query statement includes a target indicator defined in the indicator platform; A determination unit, which determines the indicator model where the target indicator is located and constructs a target database for the indicator model; A conversion unit, converting the indicator query statement into a target database query statement supported by the target database; The processing unit sends the target database query statement to the target database, queries the target database for target data corresponding to the target indicator, and returns the queried target data to the data analysis system.

8. A data processing device, applied to a data analysis system corresponding to an indicator platform, wherein the indicator platform provides a customized database connection interface, wherein the customized database connection interface is an interface for modifying the database query logic inside the database connection interface provided by any database into the indicator query logic of the indicator platform; the device comprises: An acquisition unit is configured to acquire a generated indicator query statement for a target database; wherein the indicator query statement includes a target indicator defined in the indicator platform; A replacement unit replaces the database connection interface called by the indicator query statement for connecting to the target database with the custom database connection interface, and forwards the indicator query statement to the indicator platform through the custom database connection interface; A receiving unit receives the target data returned by the indicator platform; the target data is the indicator data corresponding to the target indicator queried from the target database after the indicator platform converts the indicator query statement into a target database query statement supported by the target database.

9. A computer-readable storage medium, comprising: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data processing method as described in any one of claims 1-5 and / or 6.

10. A computing device comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the executable instructions to implement the data processing method as described in any one of claims 1-5 and / or 6.

Images