Data query method, device, electronic device and storage medium

Abstract

The present disclosure relates to a data query method, device, electronic device and storage medium, the data query method comprising: obtaining a target query request for querying data in an original data set; deleting enumerable dimension objects from the aggregate object; generating filter conditions based on the enumeration values ​​of the enumerable dimension objects in the aggregate object; and executing the aggregation rule for the data filtered out by each filter condition to determine the data query result of the target query request. The data query method, device, electronic device and storage medium disclosed in the present disclosure can solve the problem of large storage volume caused by performing data statistics for different dimensions separately, and can automatically rewrite the execution plan of the data query, simplify the data query process, and improve the query efficiency.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data management, and in particular to a data query method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of information technology, the storage volume of information data is constantly increasing, and the demand for data management and analysis is also spreading across various application scenarios. In the process of data management and analysis, it is usually necessary to perform statistical queries on data objects, in which the original data set of the data object needs to be obtained, so as to perform statistics on the data object from the original data set and output the query results.

[0003] In some application scenarios, it may be necessary to perform statistical queries on the same data object in different dimensions. In this regard, in existing data query methods, it is usually necessary to establish an original data set for each dimension, and then perform statistics on each dimension separately. However, in reality, the original data sets of different dimensions may have a certain degree of overlap. In this case, establishing an original data set for each dimension may result in a large amount of storage occupied by the data query, which is not conducive to improving storage efficiency. Summary of the invention

[0004] The present disclosure provides a data query method, device, electronic device and storage medium to at least solve the problem in the related art that data statistics are performed for different dimensions during the data query process, resulting in a large storage volume. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a data query method is provided, the data query method comprising: obtaining a target query request for querying data in an original data set, wherein the target query request carries an aggregation instruction, the aggregation instruction specifies an aggregation object and an aggregation rule, and the aggregation rule is used to group the query object of the target query request according to the data of the aggregation object; deleting an enumerable dimension object from the aggregation object, wherein the enumerable dimension object refers to an aggregation object with an enumeration value specified in the aggregation instruction; generating a filtering condition based on the enumeration value of the enumerable dimension object in the aggregation object, wherein the filtering condition is used to filter the data in the original data set; executing the aggregation rule for the data filtered out by each filtering condition to determine the data query result of the target query request.

[0006] Optionally, the data query method further includes: parsing the original data set to determine the aggregation type of the original data set; if the aggregation instruction satisfies a preset condition corresponding to the determined aggregation type, executing the step of deleting the enumerable dimension object from the aggregation object.

[0007] Optionally, the step of parsing the original data set and determining the aggregation type of the original data set includes: identifying an aggregation operator of a node tree of the original data set; and determining the aggregation type of the original data set according to the aggregation operator.

[0008] Optionally, the aggregation operator includes a logical aggregation operator, the aggregation type includes infinite stream aggregation, and the preset conditions corresponding to the infinite stream aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition contains the enumeration value of the enumerable dimension object, and the aggregation rule includes a deduplication aggregation rule, wherein the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregate object, wherein the infinite stream aggregation refers to real-time aggregation of all data in the original data set.

[0009] Optionally, the aggregation operator includes a logical window aggregation operator, the aggregation type includes window aggregation, and the preset conditions corresponding to the window aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of the enumerable dimension object, the aggregation rule includes a deduplication aggregation rule, and the window function in the logical window aggregation operator is a row semantic function, and the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, wherein the window aggregation refers to aggregating the data in the original data set that is within a preset time window and / or a preset count window.

[0010] Optionally, the step of generating filtering conditions based on the enumeration values ​​of the enumerable dimension objects in the aggregate object includes: determining each enumeration value of the enumerable dimension object as a filtering condition, wherein there is one enumerable dimension object; or, combining the enumeration values ​​of different enumerable dimension objects, and determining each combination of the enumeration values ​​as a filtering condition, wherein there are multiple enumerable dimension objects.

[0011] Optionally, for the data filtered out by each filtering condition, executing the aggregation rule, the step of determining the data query result of the target query request includes: adding a null value aggregation rule to the aggregation rule, wherein the null value aggregation rule is used to return a null value indicator in the execution result of the aggregation rule after adding the null value aggregation rule when the data filtered out by the filtering condition is a null value; for the data filtered out by each filtering condition, executing the aggregation rule after adding the null value aggregation rule, to determine the data query result of the target query request.

[0012] Optionally, for the data filtered out by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed, and the step of determining the data query result of the target query request also includes: for the data filtered out by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed to obtain the execution result for each filtering condition, wherein each execution result includes a null value data column generated according to the null value aggregation rule; filtering out the execution results in which the null value data column in the execution result includes a null value indicator to obtain the filtered execution result; deleting the data column generated according to the null value aggregation rule from the filtered execution result to determine the data query result of the target query request.

[0013] Optionally, the step of executing the aggregation rule for the data filtered out by each filtering condition and determining the data query result of the target query request includes: executing the aggregation rule for the data filtered out by each filtering condition to obtain an execution result having one or more column data; converting the one or more column data into one or more row data, and outputting the one or more row data as the data query result of the target query request.

[0014] Optionally, the aggregate object includes a non-enumerable dimension object and one or more enumerable dimension objects, wherein the non-enumerable dimension object refers to an aggregate object for which no enumeration value is specified in the aggregation instruction, and the aggregation rule is used to group the query objects of the target query request according to the aggregate objects on one or more dimensions.

[0015] According to a second aspect of an embodiment of the present disclosure, a data query device is provided, comprising: an acquisition unit, configured to acquire a target query request for querying data in an original data set, wherein the target query request carries an aggregation instruction, the aggregation instruction specifies an aggregation object and an aggregation rule, and the aggregation rule is used to group the query object of the target query request according to the data of the aggregation object; a deletion unit, configured to delete an enumerable dimension object from the aggregation object, wherein the enumerable dimension object refers to an aggregation object with an enumeration value specified in the aggregation instruction; a generation unit, configured to generate a filtering condition based on the enumeration value of the enumerable dimension object in the aggregation object, wherein the filtering condition is used to filter the data in the original data set; a determination unit, configured to execute the aggregation rule for the data filtered out by each filtering condition, and determine the data query result of the target query request.

[0016] Optionally, the data query device also includes a parsing unit, which is configured to: parse the original data set to determine the aggregation type of the original data set, and if the parsing unit determines that the aggregation instruction satisfies a preset condition corresponding to the determined aggregation type, the deletion unit executes the step of deleting the enumerable dimension object from the aggregate object.

[0017] Optionally, the parsing unit is further configured to: identify an aggregation operator of the node tree of the original data set; and determine an aggregation type of the original data set according to the aggregation operator.

[0018] Optionally, the aggregation operator includes a logical aggregation operator, the aggregation type includes infinite stream aggregation, and the preset conditions corresponding to the infinite stream aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition contains the enumeration value of the enumerable dimension object, and the aggregation rule includes a deduplication aggregation rule, wherein the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregate object, wherein the infinite stream aggregation refers to real-time aggregation of all data in the original data set.

[0019] Optionally, the aggregation operator includes a logical window aggregation operator, the aggregation type includes window aggregation, and the preset conditions corresponding to the window aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of the enumerable dimension object, the aggregation rule includes a deduplication aggregation rule, and the window function in the logical window aggregation operator is a row semantic function, and the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, wherein the window aggregation refers to aggregating the data in the original data set that is within a preset time window and / or a preset count window.

[0020] Optionally, the generation unit is further configured to: determine each enumeration value of an enumerable dimension object as a filtering condition, where there is one enumerable dimension object; or, combine enumeration values ​​of different enumerable dimension objects, and determine each combination of enumeration values ​​as a filtering condition, where there are multiple enumerable dimension objects.

[0021] Optionally, the determination unit is further configured to: add a null value aggregation rule to the aggregation rule, wherein the null value aggregation rule is used to return a null value indicator in the execution result of the aggregation rule after adding the null value aggregation rule when the data filtered by the filtering condition is a null value; for the data filtered by each filtering condition, execute the aggregation rule after adding the null value aggregation rule to determine the data query result of the target query request.

[0022] Optionally, the determination unit is further configured to: execute the aggregation rule after adding the null value aggregation rule for the data filtered out by each filtering condition to obtain the execution result for each filtering condition, wherein each execution result includes a null value data column generated according to the null value aggregation rule; filter out the execution results of the null value indicator included in the null value data column in the execution result to obtain the filtered execution result; delete the data column generated according to the null value aggregation rule from the filtered execution result to determine the data query result of the target query request.

[0023] Optionally, the determination unit is further configured to: execute the aggregation rule for the data filtered out by each filtering condition to obtain an execution result having one or more column data; convert the one or more column data into one or more row data, and output the one or more row data as the data query result of the target query request.

[0024] Optionally, the aggregate object includes a non-enumerable dimension object and one or more enumerable dimension objects, wherein the non-enumerable dimension object refers to an aggregate object for which no enumeration value is specified in the aggregation instruction, and the aggregation rule is used to group the query objects of the target query request according to the aggregate objects on one or more dimensions.

[0025] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor executable instructions, when executed by the processor, prompt the processor to execute a data query method according to an embodiment of the present disclosure.

[0026] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor, the processor is enabled to execute a data query method according to an embodiment of the present disclosure.

[0027] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement a data query method according to an embodiment of the present disclosure.

[0028] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0029] Based on the aggregation instruction, the filtering conditions of the aggregation rules can be rewritten for the enumerable dimension objects in the aggregation object, and the aggregation rules can be executed based on the rewritten filtering conditions. In this way, on the one hand, the original data sets under different enumerable dimensions can be reused by specifying the enumerable dimension objects and their enumeration values ​​in the aggregation instruction; on the other hand, by generating new filtering conditions based on the enumeration values, the execution plan of the data query can be automatically rewritten without the user having to manually write each enumeration value into the filter parameters, thereby simplifying the data query process and improving query efficiency.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0032] Figure 1 is a schematic diagram showing an example of existing data query.

[0033] Figure 2 The figure is a flow chart of a data query method according to an exemplary embodiment.

[0034] Figure 3 The present invention is a flowchart of determining a data query result of a data query method according to an exemplary embodiment.

[0035] Figure 4 The figure is a flowchart showing a data query example according to an exemplary embodiment.

[0036] Figure 5 It is a block diagram of a data query device according to an exemplary embodiment.

[0037] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0038] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0040] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0041] It should also be noted that although the application scenario of independent visitor query is explained below as an example, it should be understood that the application scenarios of the data query method, device, electronic device and storage medium according to the present disclosure are not limited to this, and can also be applied to any other data query application scenarios.

[0042] As mentioned above, in the existing data query method, it is necessary to establish an original data set for each dimension, and then perform data statistics separately to output the data query results. Here, taking the application scenario of unique visitor query as an example, in the process of analyzing the activity of applications and / or websites, unique visitors are a commonly used measurement indicator, so it is necessary to count unique visitors.

[0043] A unique visitor (UV) is a natural person who visits and browses web pages through the Internet. Multiple visits by the same visitor are counted as one unique visitor. Therefore, it is necessary to maintain a historical visitor set to count unique visitors from the historical visitor set.

[0044] In the data analysis scenario of real-time calculation statistics, it is usually necessary to count the independent visitors of different dimensions such as each page and each product line. Since there may be correlations between each page and each product line in the application and / or website, the overlap of historical visitor sets of different dimensions such as each page and each product line may be very high. For example, visitors who visit page A are likely to visit page B. In this case, if independent visitors are counted for the dimensions of page A and page B respectively, it is necessary to maintain the historical visitor sets of page A and page B respectively, which will make the storage volume of the query job larger and is not conducive to reducing the storage cost of the query job.

[0045] In this regard, in some real-time computing engine solutions for data queries, when it is necessary to calculate independent visitors from different dimensions, it is necessary to use a separate aggregation rule and a filtering condition corresponding to the aggregation rule for each dimension, so as to execute the aggregation rule based on the filtering condition to count the independent visitors in each dimension.

[0046] For example, when the product line "product" dimension includes multiple dimension values ​​"KS, "NEBULA" and "OTHER", when performing statistical queries on the independent visitor "device_id" based on the historical visitor set "source_table", the aggregation rule for calculating the number of non-duplicate results can be used as an example. For example, the Count Distinct function can be used in the SQL statement to calculate the number of non-duplicate results. Different filter parameters need to be used in each aggregation rule to limit different dimension values. Specifically, it is necessary to set aggregation rules for each dimension value "KS, "NEBULA" and "OTHER" and set filtering conditions for each aggregation rule. For example, write filtering parameters in the filtering operator of the aggregation rule. In one example, the filtering operator is the "WHERE" function. As an example, the specific query statement can be as follows:

[0047] "SELECT

[0048] TMP.product,

[0049] T1.entry_source,

[0050] TMP.uv

[0051] FROM(

[0052] --Use different filters in each Distinct aggregation function to limit different pages--

[0053] SELECT

[0054] entry_source,

[0055] COUNT(DISTINCT device_id)FILTER(WHERE product='KS')AS uv_1,

[0056] COUNT(DISTINCT device_id)FILTER(WHERE product='NEBULA')AS uv_2,

[0057] COUNT(DISTINCT device_id)FILTER(WHERE product='OTHER')AS uv_3

[0058] FROM source_table

[0059] GROUP BY entry_source

[0060] )T1,LATERAL TABLE(

[0061] --toMultipleRow converts one row to multiple rows--

[0062] toMultipleRow(uv_1,uv_2,uv_3))

[0063] AS TMP(product,uv)”.

[0064] As can be seen from the preceding query statements, it is necessary to write the CountDistinct function as an aggregation rule for each dimension value "KS", "NEBULA", and "OTHER" of the product line "product" dimension. In this way, in a computing engine such as the Flink SQL optimizer, these aggregation rules can be automatically identified and the historical visitor set can be automatically reused.

[0065] Specifically, Figure 1 The calculation process of the above data query example is shown as follows: Figure 1 As shown in the figure, the historical visitor set can be used to perform aggregation calculations on the different dimension values ​​"KS", "NEBULA" and "OTHER" of the product line "product". After the calculation is completed, the corresponding statistical results are output, such as Figure 1As shown, the number of independent visitors with the dimension value "KS" is 3 (i.e., user_id1, user_id2, and user_id3, respectively), the number of independent visitors with the dimension value "NEBULA" is 3 (i.e., user_id1, user_id2, and user_id4, respectively), and the number of independent visitors with the dimension value "OTHER" is 2 (i.e., user_id1 and user_id2, respectively).

[0066] However, in such a query method, it is necessary to manually rewrite the structured query language (SQL) to write the enumeration values ​​of multiple dimensions into the filter parameters of aggregation rules such as the Count Distinct function. The rewriting cost is high, especially when there are many enumeration values ​​under one dimension (for example, it is necessary to count the unique visitors of dozens of pages) or when there are multiple enumerable dimensions.

[0067] In addition, it can be seen from the above query statements that for the aggregate results, a custom table function "LATERALTABLE" is also required to convert columns to rows before data output, so as to output a usable data table.

[0068] The following will describe a data query method, a data query device, an electronic device, a computer-readable storage medium, and a computer program product according to exemplary embodiments of the present disclosure with reference to the accompanying drawings to solve at least one of the above problems.

[0069] Figure 2 This is a flow chart of a data query method according to an exemplary embodiment. It should be noted that the execution subject of the data query method according to the exemplary embodiment of the present disclosure may be a server, and the server may be loaded with a query statement (such as SQL) calculation engine, for example. However, the present disclosure does not specifically limit the execution subject of the above data query method, which may be any type of electronic device capable of managing data.

[0070] like Figure 2 As shown, a data query method according to an exemplary embodiment of the present disclosure may include the following steps:

[0071] In step S11 , a target query request for query data in an original data set may be obtained.

[0072] In this step, the original data set may be a data set / table in a database. For example, the original data set may be stored as a mapping state of different keys (DistinctKey MapState), and the mapping state of different keys may be a data structure used to store different key values ​​inside the stream computing engine. Taking the application scenario of counting the number of unique visitors as an example, the mapping state may be used to store a historical visitor set.

[0073] The target query request may be information / instructions sent by a user to a server to indicate the data to be queried. For example, the target query request may be an SQL query statement.

[0074] The target query request may carry an aggregation instruction, which is used to perform an aggregation operation on the query object of the data query, wherein the aggregation operation may refer to grouping / counting the data of the query object (e.g., result column) according to the data of one or more aggregation objects (e.g., data columns) in combination with a predetermined aggregation rule. In an SQL query statement, the aggregation instruction may be, for example, an aggregation statement including an aggregation operator, such as a Groupby statement.

[0075] Here, the aggregation rule may be to calculate the number of non-duplicate results of the query object in the data of one or more aggregated objects, for example, the number of non-duplicate results of the query may be realized by using an aggregation function such as a Count Distinct function. The aggregated object may be, for example, a field in the original data set, for example, a data column field. As an example, in an SQL query statement, the aggregated object may be an aggregation key (GroupKey). The filtering condition may be used to filter the field value of the aggregated object, and the target data filtered out from the original data set for the aggregated object may be obtained by using the filtering condition. When executing the aggregation rule, an aggregation operation is performed on the target data. The query object may be a data field requested by the target query request, which is generally carried in the target query request, for example, a result column field. Taking the above-mentioned application scenario of independent visitor query as an example, the original data set / table may be a historical visitor set, the aggregated object may be a product line dimension, a page dimension, the filtering condition may be each product line in the product line dimension, each page in the page dimension, and the query object may be an independent visitor. In one embodiment, the target query request includes a query object and an aggregation instruction.

[0076] The aggregation instruction carried in the target query request can specify the aggregation object and the aggregation rule. The aggregation rule can be used to group the query object of the target query request according to the data of the aggregation object. Here, the aggregation rule can be used to group the data of the query object according to the data of the aggregation object on one dimension or multiple dimensions. Therefore, the aggregation rule can be a one-dimensional aggregation rule or a multi-dimensional aggregation rule. For example, the one-dimensional aggregation rule can be implemented using the Count Distinct aggregation function, and the multi-dimensional aggregation rule can be implemented using aggregation functions such as the Cube function, the Rollup function, and the GroupSets function. It can be seen that since the aggregation rule can be a one-dimensional or multi-dimensional aggregation rule, the data query method according to the present disclosure is applicable to both ordinary one-dimensional aggregation scenarios and multi-dimensional aggregation scenarios. The applicable scenarios are wide and the universality is strong. In addition, the initial filtering condition can also be specified in the aggregation instruction. The initial filtering condition can be, for example, the field value of the aggregation object.

[0077] According to an exemplary embodiment of the present disclosure, an aggregate object may include an enumerable dimension object, where an enumerable dimension object refers to an aggregate object with an enumeration value specified in an aggregation instruction, and an initial filtering condition may include the enumeration value of the enumerable dimension object. An aggregate object may include one or more enumerable dimension objects.

[0078] Specifically, an enumerable dimension object has a specified finite number of enumeration values. Taking the application scenario of independent visitor query as an example, the product line "product" can be an enumerable dimension object, which can have enumeration values ​​"KS", "NEBULA" and "OTHER". Here, each enumerable dimension object can have one or more enumeration values.

[0079] In addition, the aggregate object may also include a non-enumerable dimension object, wherein a non-enumerable dimension object refers to an aggregate object for which no enumeration value is specified in the aggregation instruction.

[0080] In one example, the aggregation object may include an enumerable dimension object, and the aggregation rule may be implemented using a one-dimensional aggregation function, for example, the aggregation rule is written as a one-dimensional aggregation function, and when the one-dimensional aggregation function is executed, the aggregation rule is implemented. The aggregation instruction may be used to execute the aggregation rule to calculate the query object of the target query request based on the data of an enumerable dimension object.

[0081] Taking the application scenario of independent visitor query as an example, the independent visitors "device_id" of an enumerable dimension object product line "product" and a non-enumerable dimension object "entry_source" can be counted respectively. The enumeration values ​​of the enumerable dimension object product line can be "KS", "NEBULA" and "OTHER". Filter conditions can be set for aggregation rules. For example, the filter operator "WHERE" can be introduced in the SQL statement, and the enumeration values ​​"KS", "NEBULA" and "OTHER" can be reflected in the filter conditions.

[0082] In another example, the aggregation object may include multiple enumerable dimension objects, the aggregation rule may be implemented using a one-dimensional aggregation function, and the aggregation instruction may be used to execute the aggregation rule to calculate the query object of the target query request based on the data of the multiple enumerable dimension objects.

[0083] Taking the application scenario of independent visitor query as an example, the independent visitors "device_id" of the two enumerable dimension objects product line "product" and task level "platform" and the non-enumerable dimension object "entry_source" can be counted separately. The enumeration values ​​of the product line and the task level can be determined in advance. For example, the enumeration values ​​of the product line can be "KS", "NEBULA" and "OTHER", and the enumeration values ​​of the level can be "P1", "P2" and "P3". Similarly, filtering conditions can be set for the aggregation rules. For example, the filtering operator "WHERE" can be introduced in the SQL statement, and the enumeration values ​​"KS", "NEBULA" and "OTHER" and "P1", "P2" and "P3" can be reflected in the filtering conditions.

[0084] The above two examples describe the case where the aggregation rule includes a one-dimensional aggregation function. In another example, the aggregation rule may include a multidimensional aggregation function, and the aggregation object includes one or more enumerable dimension objects and one or more non-enumerable dimension objects. The aggregation instruction can be used to execute the aggregation rule to calculate the query object of the target query request based on the data of the one or more enumerable dimension objects and the non-enumerable dimension objects.

[0085] Taking the application scenario of independent visitor query as an example, the independent visitors of each product line "product" and each traffic source "entry_source" can be analyzed and counted in multiple dimensions. The enumeration values ​​of the product line "product" can be "KS", "NEBULA" and "OTHER". Similarly, filter conditions can be set for aggregation rules. For example, the filter operator "WHERE" can be introduced in the SQL statement, and the enumeration values ​​"KS", "NEBULA" and "OTHER" can be reflected in the filter conditions. Here, the product line "product" and the traffic source "entry_source" can be multi-dimensionally aggregated, that is, the independent visitors counted in the two dimensions of the product line "product" and the traffic source "entry_source" will be added to the aggregation results. For example, the statistical results in two dimensions can be added by introducing the "GROUPING_ID" function in the SQL statement.

[0086] As described in step S11 above, according to an exemplary embodiment of the present disclosure, by specifying the aggregation object and aggregation rule in the aggregation instruction of the target query request, the data query process can be simplified. Figure 1 In the described example, according to the existing data query method, it is necessary to construct the aggregation rule Count function and the corresponding filter parameter "FILTER(WHERE)" for the enumerated values ​​"KS", "NEBULA" and "OTHER" of the enumerable dimension object product line "product", respectively. According to an exemplary embodiment of the present disclosure, it is only necessary to specify the enumeration value of the enumerable dimension object in the aggregation instruction of the target query request, without the need to construct aggregation rules and filtering conditions for these enumeration values ​​one by one. In this way, the data query statement can be simplified and the data query efficiency can be improved.

[0087] In step S12, the enumerable dimension object may be deleted from the aggregate object.

[0088] In this step, there may be one or more enumerable dimension objects, and each enumerable dimension object may have one or more enumeration values.

[0089] Here, taking the aggregation statement in the SQL query statement as an example, the aggregation object can be an aggregation key (GroupKey), and the enumerable dimension column can be deleted from the original aggregation key. In this example, in step S12, the projection operator of the SQL query statement can also be rewritten, and the enumeration value of the enumerable dimension object in the aggregation object / the combination of enumeration values ​​between enumerable dimension objects can be added to the projection operator of the aggregation statement. Specifically, when there is one enumerable dimension object, each enumeration value of the enumerable dimension object is added to the projection operator of the aggregation statement; when there are multiple enumerable dimension objects, the enumeration values ​​of different enumerable dimension objects are combined, and each combination of enumeration values ​​is added to the projection operator of the aggregation statement.

[0090] For example, assuming that there are M enumerable dimension objects, and the number of enumeration values ​​of the M enumerable dimension objects is N1, N2, N3 to NM, then a value is selected from each enumerable dimension object for combination to obtain all possible enumeration value combinations between enumerable dimension objects, and there are N combinations in total, where N = N1×N2×N3...×NM. All possible enumeration value combinations between enumerable dimension objects can be added to the projection operator.

[0091] In addition, according to an exemplary embodiment of the present disclosure, the data query method may further include: S101, parsing the original data set to determine the aggregation type of the original data set; S103, if the aggregation instruction meets the preset conditions corresponding to the determined aggregation type, executing step S12.

[0092] In step S101, an aggregation operator of a node tree of an original data set is identified; and according to the aggregation operator, an aggregation type of the original data set is determined.

[0093] In one example, the aggregation operator may include a logical aggregation operator (LogicalAggregate operator), for example, the node tree includes a logical aggregation operator. The aggregation type may include infinite stream aggregation, wherein infinite stream aggregation refers to real-time aggregation of all data in the original data set.

[0094] In an example where the aggregation operator includes a logical aggregation operator, the preset conditions corresponding to the infinite stream aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of an enumerable dimension object, and the aggregation rule includes a deduplication aggregation rule. Here, the deduplication aggregation rule may refer to removing duplicate values ​​in the data of the aggregation object, for example, the deduplication aggregation rule may be a COUNT DISTINCT aggregation function.

[0095] In another example, the aggregation operator may include a logical window aggregation operator (LogicalWindowAggregate operator), for example, the node tree includes a logical window aggregation operator. The aggregation type may include window aggregation, which refers to aggregating data in a preset time window and / or a preset count window in the original data set.

[0096] In an example where the aggregation operator includes a logical window aggregation operator, preset conditions corresponding to window aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of an enumerable dimension object, the aggregation rule includes a deduplication aggregation rule, and the window function in the logical window aggregation operator is a function that performs operations on rows of data in a data table, such as a row semantic function.

[0097] Generally speaking, window functions can include row semantics functions and set semantics functions. For window functions with row semantics, which window the input data is divided into depends only on the current input data, and has nothing to do with the historical input data of the window. For window functions with set semantics, which window the input data is divided into depends not only on the current input data, but also on the historical input data of the window. Since in the data query method according to the present disclosure, by generating filter conditions based on enumeration values, it is possible to reuse an original data set to perform aggregate calculations on different enumeration values ​​of enumerable dimensions, it is different from the reference Figure 1 Compared with the described method that requires multiple original data sets to perform aggregation calculations, the method according to the exemplary embodiment of the present invention needs to rewrite the data distribution mechanism of the aggregation instruction for the original data set. Therefore, the original data set for each window will change. In this case, the window needs to be independent of the historical input data. Therefore, the window function of the embodiment of the present invention selects the row semantics function, which enables the data query method according to the present invention to better adapt to the aggregation calculation in the window aggregation scenario and avoid query errors caused by the window function with set semantics in the query.

[0098] According to the data query method of the exemplary embodiment of the present disclosure, data query is optimized by specifying the aggregation object and the aggregation rule. The process is not limited by the aggregation type. Therefore, the method is applicable to both the aggregation on the finite window and the aggregation on the infinite stream, and has strong universality. On this basis, in order to improve the efficiency of data query and respond to the query request that cannot be executed in time, in the above steps S101 to S103, preset conditions for executing the data query method are added for the finite window aggregation and the infinite stream aggregation respectively. The preset conditions corresponding to the determined aggregation type can be, for example, the optimization rules added in the SQL optimizer of the real-time computing engine. When the optimization rules are met, it can be considered that the data query can be executed based on the aggregation instruction. In this regard, in step S103, if the aggregation instruction satisfies the preset conditions corresponding to the determined aggregation type, step S12 can be executed. If the aggregation instruction does not meet the preset conditions corresponding to the determined aggregation type, it can be considered that the data query cannot be executed based on the aggregation instruction, so that the query can be terminated. In this way, by judging whether the aggregation instruction meets the corresponding preset conditions, it is possible to pre-judge whether the data query method can be executed in the current aggregation scenario. When the preset conditions are not met, the data query can be terminated in time, thereby improving the response speed and query efficiency of the data query.

[0099] In step S13, a filter condition is generated based on the enumeration value of the enumerable dimension object in the aggregate object. Here, the filter condition can be used to filter the data in the original data set.

[0100] In one case, there may be one enumerable dimension object, and in step S13, each enumeration value of the enumerable dimension object may be determined as a filtering condition.

[0101] Specifically, taking statement (1) in the application scenario of independent visitor query described above as an example, the enumerable dimension object is the product line "product". In this case, each enumeration value "KS", "NEBULA" and "OTHER" of the enumerable dimension object can be used as a separate filtering condition, so that in step S14 described below, the aggregation rule is executed for each filtering condition to obtain the aggregation result for each filtering condition.

[0102] In another case, there may be multiple enumerable dimension objects. In step S13, the enumeration values ​​of each enumerable dimension object may be combined, and each combination of the enumeration values ​​may be determined as a filtering condition.

[0103] Specifically, as described in step S12, assuming that there are M enumerable dimension objects, the number of enumeration values ​​of the M enumerable dimension objects is N1, N2, N3 to NM respectively, so that there are N combinations of enumeration values ​​between different enumerable dimension objects, where N = N1×N2×N3...×NM. Accordingly, a filter condition can be generated based on each of the N enumeration value combinations. Here, in the process of generating a filter condition based on each enumeration value combination, the filter condition can be generated by logically combining and connecting the enumeration values. For example, the enumeration values ​​can be connected using the AND condition of the logical AND operation, and the connected enumeration value combination can be used as the filter condition.

[0104] As described in the above two cases, according to the data query method of the exemplary embodiment of the present disclosure, different filter conditions can be generated according to the number of enumerable dimension objects.

[0105] In step S14, the aggregation rule may be executed for the data filtered out by each filtering condition to determine the data query result of the target query request.

[0106] As an example, see Figure 3 , step S14 may include the following steps:

[0107] In step S141, a null value aggregation rule is added to the aggregation rule to rewrite the aggregation rule. Here, the null value aggregation rule is used to return a null value indicator in the execution result of the aggregation rule after the null value aggregation rule is added when the data filtered by the filter condition is null.

[0108] In this way, for the data filtered out by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed to determine the data query result of the target query request.

[0109] Specifically, in step S142, for the data filtered by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed to obtain the execution result for each filtering condition, where each execution result includes a null value data column generated according to the null value aggregation rule. Here, when the data filtered by the filtering condition is a null value, the null value data column may include a null value indicator.

[0110] As an example, the null value aggregation rule may be implemented by a Count aggregation function. Specifically, a Count aggregation function may be added to the aggregation process of each filter condition generated in step S13 to filter out empty data sets (also called empty buckets) with no data.

[0111] In step S143, the execution results including the null value indicator in the null value data column are filtered out to obtain the filtered execution results.

[0112] Specifically, in this step, the null values ​​of the data can be filtered out based on the null value indicator, so that the null value results in the final query results can be avoided. Here, in the example where the null value aggregation rule is the Count aggregation function, a filtering operator can be constructed to filter out the empty buckets whose Count value is 0 in the data.

[0113] In step S144, the data columns generated according to the null value aggregation rule are deleted from the filtered execution results to determine the data query result of the target query request.

[0114] Since the null value condition is added in step S141 to filter out the null values ​​of the data and avoid the null value results in the final query results, and the result columns generated based on the null value aggregation rule do not need to be reflected in the final query results, after filtering out the execution results including the null value indicator, in step S144, the result columns generated based on the null value aggregation rule in the remaining execution results are deleted to delete the result columns used to filter the null value results, thereby maintaining the simplicity of the output query results. Here, in the example of the SQL query statement, a projection operator can be constructed to trim the result columns generated based on the null value condition, such as the result column of the Count aggregation function.

[0115] In addition, according to an exemplary embodiment of the present disclosure, in step S14, conversion of data form may also be performed.

[0116] Specifically, step S14 may also include: executing an aggregation rule for the data filtered by each filter condition to obtain an execution result having one or more column data; converting the one or more column data into one or more row data, and outputting the one or more row data as the data query result of the target query request. Here, in the example of the SQL query statement, a column-to-row operator may be constructed based on step S14.

[0117] As reference Figure 1 In the described query statement, a custom table function is required to convert columns to rows before data output, so as to output a usable data table. In this regard, in an exemplary embodiment of the present disclosure, based on the received aggregation instruction, the conversion of columns to rows can be automatically performed on the execution result of the aggregation instruction without the need to customize the input of the column-to-row instruction.

[0118] Figure 4 A flowchart of a data query example according to an exemplary embodiment of the present disclosure in an application scenario of independent visitor query is shown.

[0119] like Figure 4As shown, a historical visitor set can be assigned to multiple enumeration values ​​"KS", "NEBULA" and "OTHER" of the enumerable dimension object product line "product", for example, the historical visitor set can be stored in Distinct KeyMapState. In other words, the historical visitor set as the original data set can be reused for different enumeration values, including all user IDs, namely {'user_id1', 'user_id2', 'user_id3', 'user_id4'}. In the calculation process, the filter conditions can be generated based on the enumeration value, and the aggregation rules can be executed for the filter conditions "product = 'KS'", "product = 'NEBULA'" and "product = 'OTHER'" respectively to generate the execution results for the three filter conditions, that is, the number of independent visitors with the value "KS" is 3, the number of independent visitors with the value "NEBULA" is 3, and the number of independent visitors with the value "OTHER" is 2. Therefore, the output query result is expressed as (3,3,2). Since the execution result is in the form of a row of multiple columns, the query result can be converted from row to column by the row-to-column operator, so that the final query result can be determined, that is:

[0120] "'KS',3

[0121] 'NEBULA',3

[0122] 'OTHER',2".

[0123] exist Figure 1 In the existing query method, it is necessary to use the historical visitor set to perform aggregation calculations for the different dimension values ​​"KS", "NEBULA" and "OTHER" of the product line "product", which requires multiple historical visitor sets. Figure 4 In the query method of the exemplary embodiment of the present disclosure described, a historical visitor set can be assigned to multiple enumeration values ​​"KS", "NEBULA" and "OTHER" of the enumerable dimensional object product line "product", and then the historical visitor set can be used for aggregation calculation. In other words, aggregate calculation can be performed on query objects under multiple enumeration values ​​while using one historical visitor set, which reduces the amount of storage occupied during the data query process.

[0124] Refer to above Figure 4An example is described when there is only one enumerable dimension object. However, as described above, in the method according to the present disclosure, there may be multiple enumerable dimension objects. In this case, the enumeration values ​​of the various enumerable dimension objects may be combined, and the combined enumeration values ​​may be used as filtering conditions to execute aggregation rules, so that the execution of the method according to the present disclosure is not limited by the number of enumerable dimension objects, and queries may be performed on a single enumerable dimension object or on multiple enumerable dimension objects. An exemplary embodiment of a data query method according to the present disclosure is described above. In this method, an enumerable dimension object may be deleted from an aggregation object, and then its enumeration value may be added to the filtering condition. In this way, the overall aggregation result may be kept unchanged. However, since the enumeration dimension object is deleted from the aggregation object, the original data set in the aggregation calculation may be reduced, and aggregation calculations may be performed on different enumeration values ​​of the enumerable dimension by reusing an original data set.

[0125] In addition, since in the above process, the filtering condition is generated based on the enumeration value, when executing the aggregation rule, each enumeration value can be automatically aggregated based on the filtering condition. Compared with the existing query method in which the aggregation rule for each enumeration value needs to be manually added, this allows the SQL optimizer of the real-time computing engine to automatically rewrite the execution plan of the data query without the user having to manually write each enumeration value into the filter parameter, thereby simplifying the data query process and improving query efficiency.

[0126] Figure 5 is a block diagram of a data query device according to an exemplary embodiment. Figure 5 The device includes an acquisition unit 100, a deletion unit 200, a generation unit 300 and a determination unit 400.

[0127] The acquisition unit 100 can be configured to acquire a target query request for querying data in an original data set, wherein the target query request carries an aggregation instruction, the aggregation instruction specifies an aggregation object and an aggregation rule, and the aggregation rule is used to group the query object of the target query request according to the data of the aggregation object.

[0128] The deleting unit 200 may be configured to delete an enumerable dimension object from an aggregate object, wherein the enumerable dimension object refers to an aggregate object having an enumeration value specified in an aggregation instruction.

[0129] The generating unit 300 may be configured to generate a filtering condition based on an enumeration value of an enumerable dimension object in an aggregate object, wherein the filtering condition is used to filter data in an original data set.

[0130] The determination unit 400 may be configured to execute an aggregation rule for the data filtered out by each filtering condition, and determine a data query result of the target query request.

[0131] As an example, the data query device also includes a parsing unit 500, which can be configured to: parse the original data set to determine the aggregation type of the original data set; if the parsing unit 500 determines that the aggregation instruction satisfies a preset condition corresponding to the determined aggregation type, the deletion unit 200 executes the step of deleting the enumerable dimension object from the aggregate object.

[0132] As an example, the parsing unit 500 may also be configured to: identify an aggregation operator of a node tree of the original data set; and determine an aggregation type of the original data set according to the aggregation operator.

[0133] Specifically, the aggregation operator may include a logical aggregation operator, the aggregation type includes infinite stream aggregation, and the preset conditions corresponding to the infinite stream aggregation may include: the aggregation instruction includes an initial filtering condition, the initial filtering condition contains an enumeration value of an enumerable dimension object, and the aggregation rule includes a deduplication aggregation rule, wherein the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, wherein the infinite stream aggregation refers to real-time aggregation of all data in the original data set.

[0134] The aggregation operator may also include a logical window aggregation operator, the aggregation type includes window aggregation, and the preset conditions corresponding to the window aggregation may include: the aggregation instruction includes an initial filtering condition, the initial filtering condition contains an enumeration value of an enumerable dimension object, the aggregation rule includes a deduplication aggregation rule, and the window function in the logical window aggregation operator is a row semantic function, and the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, wherein window aggregation refers to aggregating data in the original data set that is within a preset time window and / or a preset count window.

[0135] As an example, the generation unit 300 can also be configured to: determine each enumeration value of an enumerable dimension object as a filtering condition, where there is one enumerable dimension object; or, combine the enumeration values ​​of different enumerable dimension objects, and determine each combination of the enumeration values ​​as a filtering condition, where there are multiple enumerable dimension objects.

[0136] As an example, the determination unit 400 can also be configured to: add a null value aggregation rule to the aggregation rule to rewrite the aggregation rule, wherein the null value aggregation rule is used to return a null value indicator in the execution result of the aggregation rule after adding the null value aggregation rule when the data filtered by the filtering condition is a null value; for the data filtered by each filtering condition, execute the aggregation rule after adding the null value aggregation rule to determine the data query result of the target query request.

[0137] As an example, the determination unit 400 can also be configured to: for the data filtered out by each filtering condition, execute the aggregation rule after adding the null value aggregation rule to obtain the execution result for each filtering condition, wherein each execution result includes a null value data column generated according to the null value aggregation rule; filter out the execution results in which the null value data column in the execution result includes a null value indicator to obtain the filtered execution result; based on the filtered execution result, determine the data query result of the target query request.

[0138] As an example, the determination unit 400 may also be configured to: delete the data column generated according to the null value aggregation rule from the filtered execution result to determine the data query result of the target query request.

[0139] As an example, the determination unit 400 can also be configured to: execute the aggregation rule for the data filtered out by each filtering condition to obtain an execution result having one or more column data; convert the one or more column data into one or more row data, and output the one or more row data as the data query result of the target query request.

[0140] As an example, an aggregate object includes a non-enumerable dimension object and one or more enumerable dimension objects, wherein the non-enumerable dimension object refers to an aggregate object for which no enumeration value is specified in the aggregation instruction, and the aggregation rule is used to group the query objects of the target query request according to the aggregation objects on one or more dimensions.

[0141] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0142] Figure 6 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Figure 6 As shown, the electronic device 10 includes a processor 101 and a memory 102 for storing processor executable instructions. Here, when the processor executable instructions are executed by the processor, the processor is prompted to execute the data query method as described in the above exemplary embodiment.

[0143] As an example, the electronic device 10 is not necessarily a single device, but may be any collection of devices or circuits that can execute the above instructions (or instruction sets) individually or in combination. The electronic device 10 may also be part of an integrated control system or system manager, or may be configured as a server that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0144] In the electronic device 10, the processor 101 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not a limitation, the processor 101 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0145] The processor 101 may execute instructions or codes stored in the memory 102, wherein the memory 102 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0146] The memory 102 may be integrated with the processor 101, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 102 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 102 and the processor 101 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 101 can read files stored in the memory 102.

[0147] In addition, the electronic device 10 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 10 may be connected to each other via a bus and / or a network.

[0148] In an exemplary embodiment, a computer-readable storage medium may also be provided, and when the instructions in the computer-readable storage medium are executed by the processor, the processor is enabled to execute the data query method as described in the above exemplary embodiment. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: a read-only memory (ROM), a random access memory (RAM), a random access programmable read-only memory (PROM), an electrically erasable programmable read-only memory (EEPROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a non-volatile memory, a CD-ROM, a CD-R, a CD+R, a CD-RW, a CD+RW, a DVD-ROM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a DVD-RAM, a BD-ROM, a BD-R, a BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0149] In an exemplary embodiment, a computer program product may also be provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the data query method as described in the above exemplary embodiment is implemented.

[0150] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0151] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data query method, It is characterized in that The data query method comprises: Obtaining a target query request for querying data in an original data set, wherein the target query request carries an aggregation instruction, the aggregation instruction specifies an aggregation object and an aggregation rule, and the aggregation rule is used to group the query object of the target query request according to the data of the aggregation object; Deleting an enumerable dimension object from the aggregate object, wherein the enumerable dimension object refers to an aggregate object having an enumeration value specified in the aggregate instruction; Generate a filtering condition based on the enumeration value of the enumerable dimension object in the aggregate object, wherein the filtering condition is used to filter the data in the original data set; The aggregation rule is executed for the data filtered out by each filtering condition to determine the data query result of the target query request.

2. The data query method according to claim 1, It is characterized in that The data query method further includes: Parsing the original data set to determine an aggregation type of the original data set; If the aggregation instruction satisfies a preset condition corresponding to the determined aggregation type, a step of deleting the enumerable dimension object from the aggregation object is performed.

3. The data query method according to claim 2, It is characterized in that The step of parsing the original data set and determining the aggregation type of the original data set includes: identifying an aggregation operator of a node tree of the original data set; and determining the aggregation type of the original data set according to the aggregation operator.

4. The data query method according to claim 3, It is characterized in that The aggregation operator includes a logical aggregation operator, the aggregation type includes an infinite stream aggregation, and the preset conditions corresponding to the infinite stream aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of the enumerable dimension object, and the aggregation rule includes a deduplication aggregation rule, wherein the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, The infinite stream aggregation refers to real-time aggregation of all data in the original data set.

5. The data query method according to claim 3, It is characterized in that The aggregation operator includes a logical window aggregation operator, the aggregation type includes window aggregation, and the preset conditions corresponding to the window aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of the enumerable dimension object, the aggregation rule includes a deduplication aggregation rule, and the window function in the logical window aggregation operator is a row semantic function, and the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, The window aggregation refers to aggregating the data in the original data set that is within a preset time window and / or a preset count window.

6. The data query method according to claim 1, It is characterized in that The step of generating a filtering condition based on the enumeration value of the enumerable dimension object in the aggregate object comprises: Determine each enumeration value of an enumerable dimension object as a filtering condition, wherein there is one enumerable dimension object; or, Enumeration values ​​of different enumerable dimension objects are combined, and each combination of the enumeration values ​​is determined as a filtering condition, wherein there are multiple enumerable dimension objects.

7. The data query method according to claim 1, It is characterized in that The step of executing the aggregation rule for the data filtered out by each filtering condition to determine the data query result of the target query request includes: Adding a null value aggregation rule to the aggregation rule, wherein the null value aggregation rule is used to return a null value indicator in the execution result of the aggregation rule after the null value aggregation rule is added when the data filtered by the filtering condition is a null value; For the data filtered out by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed to determine the data query result of the target query request.

8. The data query method according to claim 7, It is characterized in that The step of executing the aggregation rule after adding the null value aggregation rule for the data filtered out by each filtering condition to determine the data query result of the target query request further includes: For the data filtered by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed to obtain the execution result for each filtering condition, wherein each execution result includes the null value data column generated according to the null value aggregation rule; Filter out the execution results whose null value data columns in the execution results include null value indicators to obtain filtered execution results; The data columns generated according to the null value aggregation rule are deleted from the execution results of the filtering to determine the data query result of the target query request.

9. The data query method according to claim 1, It is characterized in that The step of executing the aggregation rule for the data filtered out by each filtering condition to determine the data query result of the target query request includes: For the data filtered out by each filtering condition, the aggregation rule is executed to obtain an execution result having one or more columns of data; The one or more column data are converted into one or more row data, and the one or more row data are output as the data query result of the target query request.

10. The data query method according to claim 1, It is characterized in that The aggregate object includes a non-enumerable dimension object and one or more of the enumerable dimension objects, wherein the non-enumerable dimension object refers to an aggregate object for which no enumeration value is specified in the aggregation instruction, The aggregation rule is used to group the query objects of the target query request according to the aggregation objects in one or more dimensions.

11. A data query device, It is characterized in that The data query device comprises: an acquisition unit, configured to acquire a target query request for querying data in an original data set, wherein the target query request carries an aggregation instruction, the aggregation instruction specifies an aggregation object and an aggregation rule, and the aggregation rule is used to group query objects of the target query request according to the data of the aggregation object; a deleting unit configured to delete an enumerable dimension object from the aggregate object, wherein the enumerable dimension object refers to an aggregate object having an enumeration value specified in the aggregation instruction; A generating unit, configured to generate a filtering condition based on an enumerated value of an enumerable dimension object in the aggregate object, wherein the filtering condition is used to filter the data in the original data set; The determination unit is configured to execute the aggregation rule for the data filtered out by each filtering condition to determine the data query result of the target query request.

12. The data query device according to claim 11, It is characterized in that The data query device also includes a parsing unit, which is configured to: parse the original data set to determine the aggregation type of the original data set, and if the parsing unit determines that the aggregation instruction meets the preset conditions corresponding to the determined aggregation type, the deletion unit executes the step of deleting the enumerable dimension object from the aggregate object.

13. The data query device according to claim 12, It is characterized in that The parsing unit is further configured to: identify an aggregation operator of the node tree of the original data set; and determine an aggregation type of the original data set according to the aggregation operator.

14. The data query device according to claim 13, It is characterized in that The aggregation operator includes a logical aggregation operator, the aggregation type includes an infinite stream aggregation, and the preset conditions corresponding to the infinite stream aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of the enumerable dimension object, and the aggregation rule includes a deduplication aggregation rule, wherein the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, The infinite stream aggregation refers to real-time aggregation of all data in the original data set.

15. The data query device according to claim 13, It is characterized in that The aggregation operator includes a logical window aggregation operator, the aggregation type includes window aggregation, and the preset conditions corresponding to the window aggregation include: the aggregation instruction includes an initial filtering condition, the initial filtering condition includes an enumeration value of the enumerable dimension object, the aggregation rule includes a deduplication aggregation rule, and the window function in the logical window aggregation operator is a row semantic function, and the deduplication aggregation rule is used to delete duplicate values ​​in the data of the aggregation object, The window aggregation refers to aggregating the data in the original data set that is within a preset time window and / or a preset count window.

16. The data query device according to claim 11, It is characterized in that The generating unit is further configured to: Determine each enumeration value of an enumerable dimension object as a filtering condition, wherein there is one enumerable dimension object; or, Enumeration values ​​of different enumerable dimension objects are combined, and each combination of the enumeration values ​​is determined as a filtering condition, wherein there are multiple enumerable dimension objects.

17. The data query device according to claim 11, It is characterized in that The determining unit is further configured to: Adding a null value aggregation rule to the aggregation rule, wherein the null value aggregation rule is used to return a null value indicator in the execution result of the aggregation rule after the null value aggregation rule is added when the data filtered by the filtering condition is a null value; For the data filtered out by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed to determine the data query result of the target query request.

18. The data query device according to claim 17, It is characterized in that The determining unit is further configured to: For the data filtered by each filtering condition, the aggregation rule after adding the null value aggregation rule is executed to obtain the execution result for each filtering condition, wherein each execution result includes the null value data column generated according to the null value aggregation rule; Filter out the execution results whose null value data columns in the execution results include null value indicators to obtain filtered execution results; The data columns generated according to the null value aggregation rule are deleted from the execution results of the filtering to determine the data query result of the target query request.

19. The data query device according to claim 11, It is characterized in that The determining unit is further configured to: For the data filtered out by each filtering condition, the aggregation rule is executed to obtain an execution result having one or more columns of data; The one or more column data are converted into one or more row data, and the one or more row data are output as the data query result of the target query request.

20. The data query device according to claim 11, It is characterized in that The aggregate object includes a non-enumerable dimension object and one or more of the enumerable dimension objects, wherein the non-enumerable dimension object refers to an aggregate object for which no enumeration value is specified in the aggregation instruction, The aggregation rule is used to group the query objects of the target query request according to the aggregation objects in one or more dimensions.

21. An electronic device, It is characterized in that The electronic device comprises: processor; a memory for storing instructions executable by the processor, Wherein, when the processor executable instructions are executed by the processor, the processor is prompted to execute the data query method according to any one of claims 1 to 10.

22. A computer-readable storage medium, It is characterized in that When the instructions in the computer-readable storage medium are executed by a processor, the processor is enabled to execute the data query method according to any one of claims 1 to 10.

23. A computer program product comprising computer instructions, It is characterized in that When the computer instructions are executed by a processor, the data query method according to any one of claims 1 to 10 is implemented.

Images