Data query method and data query system

Abstract

The invention discloses a data query method and a data query system, which can improve the efficiency of data query. The data query method can be applied to a data query system. A data query system receives a data query request, the request including an identification of a data model and a first filter. Then, the data query system obtains configuration information of the data model according to the identifier of the data model, and the configuration information of the data model comprises an incidence relation between data tables in the data model or a field defined in the data model; and then, the data query system optimizes the first filter according to the configuration information of the data model to obtain a new filter, and then executes data query operation on the data model according to the new filter.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a data query method and a data query system. Background Art

[0002] During the data query process, the data query system can directly query the required data from the corresponding data model according to the filter configured by the user. However, the above query method may lead to a large amount of data screening or processing work, and thus result in low data query efficiency. Summary of the Invention

[0003] This application provides a data query method and a data query system, which can improve the efficiency of data query.

[0004] In a first aspect, this application provides a data query method. This method can be applied to a data query system. The data query system receives a data query request, which includes an identifier of a data model and a first filter. Then, the data query system obtains the configuration information of the data model according to the identifier of the data model, where the configuration information of the data model includes the association relationship between data tables in the data model or the fields defined in the data model. After that, the data query system optimizes the first filter according to the configuration information of the data model to obtain a new filter, and then performs a data query operation on the data model according to the new filter.

[0005] In the technical solution provided by this application, the data query system can optimize the filter configured by the user according to the configuration information of the data model, and the optimized new filter can filter the data model according to the fields of the data tables in the data model. Therefore, the technical solution provided by this application can improve the efficiency of data query.

[0006] In a possible implementation, the data query system optimizes the first filter according to the configuration information of the data model to obtain a new filter, including: the data query system splits the first filter according to the connection relationship between the fields in the first filter to obtain a second filter, and then optimizes the second filter according to the configuration information of the data model to obtain a new filter.

[0007] It should be understood that in practical applications, the first filter may be relatively complex (for example, including a large number of fields). By splitting the first filter, the complexity of the filter can be reduced (for example, including fewer fields), making it easier to optimize the filter.

[0008] In a possible implementation, the second filter includes a first field, and the configuration information of the data model includes the association relationships between the data tables in the data model. The data query system optimizes the second filter according to the configuration information of the data model to obtain a new filter, including: the data query system obtains a second field, where the second field is the first field or the source field on which the first field depends, and the source field is a field in the data table in the data model. Then, the data query system obtains the fields associated with the second field according to the second field and the association relationships between the data tables in the data model, and then replaces the first field in the second filter with the fields associated with the second field, thereby obtaining a new filter.

[0009] It should be understood that in actual applications, some fields in the data tables in the data model will be hidden at the consumer side. In this case, the user will not be able to configure a filter based on the hidden fields, and thus will not be able to directly obtain the data corresponding to the hidden fields in the data table according to the configured filter. However, the new filter obtained through the above implementation includes the above-mentioned hidden fields. Therefore, the data corresponding to the hidden fields in the data table can be directly obtained according to the new filter, thereby improving the efficiency of data query.

[0010] In a possible implementation, the second filter includes a first field, and the configuration information of the data model includes the fields defined in the data model. The data query system optimizes the second filter according to the configuration information of the data model to obtain a new filter, including: the data query system obtains the source field on which the first field depends according to the matching relationship between the first field and the fields defined in the data model, and the source field is a field in the data table in the data model. Then, the data query system replaces the first field in the second filter with the above-mentioned source field, thereby obtaining a new filter.

[0011] In a possible implementation, the fields defined in the data model include calculated fields. The data query system obtains the source field on which the first field depends according to the matching relationship between the first field and the fields defined in the data model, including: the data query system obtains the calculation expression corresponding to the matching calculated field according to the matching relationship between the first field and the calculated fields in the data model, and the calculation expression is used to indicate the relationship between the matching calculated field and the above-mentioned source field. Then, the data query system processes the first field according to the above-mentioned calculation expression to obtain the above-mentioned source field.

[0012] In a possible implementation, the fields defined in the data model include type conversion fields. The data query system obtains the source fields on which the first field depends according to the matching relationship between the first field and the fields defined in the data model, including: the data query system obtains the conversion expression corresponding to the matched type conversion field according to the matching relationship between the first field and the type conversion fields in the data model, and the conversion expression is used to indicate the relationship between the matched type conversion field and the above source fields. Then, the data query system processes the first field according to the above conversion expression to obtain the above source fields.

[0013] In the above implementation, the data query system can obtain the source fields on which the calculation field or type conversion field in the original filter depends according to the calculation field or type conversion field in the original filter, so as to obtain a new filter. Since the new filter includes the source fields on which the calculation field or type conversion field depends, the new filter can be directly used to filter the data table, thereby improving the efficiency of data query.

[0014] In a second aspect, the present application provides a data query system. The system includes an acquisition module, a filter optimization module, and a data query module. The acquisition module is used to receive a data query request, and the request includes an identifier of a data model and a first filter. The filter optimization module is used to obtain the configuration information of the data model according to the identifier of the data model, where the configuration information of the data model includes the association relationship between data tables in the data model or the fields defined in the data model. The filter optimization module is further used to optimize the first filter according to the configuration information of the data model to obtain a new filter. The data query module is used to perform a data query operation on the data model according to the new filter.

[0015] In a possible implementation, the filter optimization module is used to split the first filter according to the connection relationship between the fields in the first filter to obtain a second filter, and then optimize the second filter according to the configuration information of the data model to obtain a new filter.

[0016] In a possible implementation, the second filter includes a first field, and the configuration information of the data model includes the association relationship between data tables in the data model. The filter optimization module is used to obtain a second field, where the second field is the first field or the source field on which the first field depends, and the source field is a field in the data table in the data model. Then, according to the second field and the association relationship between the data tables in the data model, the fields associated with the second field are obtained, and the first field in the second filter is replaced with the fields associated with the second field, so as to obtain a new filter.

[0017] In a possible implementation, the second filter includes a first field, and the configuration information of the data model includes the fields defined in the data model. The filter optimization module is configured to obtain the source field on which the first field depends according to the matching relationship between the first field and the fields defined in the data model, where the source field is a field in the data table of the data model, and then replace the first field in the second filter with the above source field to obtain a new filter.

[0018] In a possible implementation, the fields defined in the data model include calculation fields. The filter optimization module is configured to obtain the calculation expression corresponding to the matching calculation field according to the matching relationship between the first field and the calculation fields in the data model, where the calculation expression is used to indicate the relationship between the matching calculation field and the above source field, and then process the first field according to the above calculation expression to obtain the above source field.

[0019] In a possible implementation, the fields defined in the data model include type conversion fields. The filter optimization module is configured to obtain the conversion expression corresponding to the matching type conversion field according to the matching relationship between the first field and the type conversion fields in the data model, where the conversion expression is used to indicate the relationship between the matching type conversion field and the above source field, and then process the first field according to the above conversion expression to obtain the above source field.

[0020] In a third aspect, the present application provides a computing device. The computing device includes a processor and a memory. The processor is configured to execute instructions stored in the memory so that the computing device executes some or all of the methods described in the foregoing first aspect and any one of its implementations.

[0021] In a fourth aspect, the present application provides a computer program product. The computer program product may be software or a program product including instructions that can run on a computing device or be stored in any available medium. When the computer program product runs on a computing device, it causes the computing device to execute some or all of the methods described in the foregoing first aspect and any one of its implementations.

[0022] In a fifth aspect, the present application provides a computer-readable storage medium. The computer storage medium includes computer program instructions. When the computer program instructions are executed by a computing device, it causes the computing device to execute some or all of the methods described in the foregoing first aspect and any one of its implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram of a data query scenario provided by the present application;

[0024] Figure 2It is a schematic flowchart of a data query method provided by this application;

[0025] Figure 3 It is a schematic diagram of a query interface provided by this application;

[0026] Figure 4 It is a schematic diagram of a data model and its configuration information provided by this application;

[0027] Figure 5 It is a schematic structural diagram of a data query system provided by this application;

[0028] Figure 6 It is a schematic structural diagram of a computing device provided by this application;

[0029] Figure 7 It is a schematic structural diagram of a computing device cluster provided by this application. Detailed implementation manners

[0030] In the data query scenario, a data model is a logical model that abstracts and encapsulates the data in the data source based on actual business requirements to provide users with a higher-level data access interface, thereby facilitating users to understand and use the data. In actual applications, some fields of the data tables in the data model will be hidden at the consumer end, making it impossible for users to directly configure filters based on the hidden fields, resulting in the need to filter the entire data table subsequently, increasing the workload of data filtering and affecting the efficiency of data query. In addition, users can also configure filters based on the fields defined in the data model (for example, calculated fields or type conversion fields). However, since the fields defined in the data model are obtained by processing the fields in the data table, when using the above filters to filter the data table, it is also necessary to process the data in the data table, which also leads to low data query efficiency.

[0031] To solve the problem of low data query efficiency, this application provides a data query method, which can be applied to a data query system. After the data query system obtains the filters configured by the user and the data model to be queried, it can optimize the filters configured by the user according to the configuration information in the data model to obtain new filters. Compared with the filters configured by the user, when performing data query operations on the data model based on the new filters, the workload of data filtering and processing can be reduced. Therefore, the technical solution provided by this application can improve the efficiency of data query.

[0032] Next, the technical solution provided by this application will be described with reference to the accompanying drawings.

[0033] Please refer to Figure 1 , Figure 1Shows a data query scenario applicable to the present application. As Figure 1 shown, this scenario includes a data query system 100 and a client 200. Among them, the data query system 100 and the client 200 are connected through a network, and this network can be a wide area network or a local area network.

[0034] The data query system 100 can be divided into a data source 101, a semantic layer 102, and a consumption layer 103. The data source 101 is used to store and manage multiple data tables. The data tables in the data source 101 include fact tables and dimension tables. For example, fact table 1, fact table 2, dimension table 1, dimension table 2, etc. in the figure. The semantic layer 102 stores multiple preset data models and configuration information of multiple preset data models. Among them, the preset data model is obtained by abstracting and encapsulating the data tables in the data source 101 based on business requirements, and can provide a higher-level data access interface, facilitating users to query the required data in a more intuitive and semantic way. The configuration information of the preset data model is the relevant information configured during the establishment of the preset data model and used to define and describe the preset data model. For example, the association relationship between multiple data tables such as fact table 1, dimension table 1, and dimension table 2 in the figure. The consumption layer 103 is used to provide an interface for users to access the data in the data source 101 through the data model.

[0035] In Figure 1 the shown data query scenario, when a user wants to query data, the user can send a data query request to the data query system 100 through the client 200. Correspondingly, the consumption layer 103 in the data query system 100 receives the data query request sent by the client 200 and sends the data query request to the semantic layer 102. The semantic layer 102 obtains the configuration information of the data model to be queried and the filter specified by the user according to the data query request, and then optimizes the filter specified by the user according to the configuration information of the data model. The semantic layer 102 also generates an SQL statement according to the optimized new filter, and then queries the data required by the user from the data source 101 according to the generated SQL statement, and then returns the queried data to the client 200 through the consumption layer 103 so that the user can obtain the required data.

[0036] In a specific implementation, the data query system 100 can be a hardware system. For example, it can be implemented through a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The above PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a system on chip (SoC), or any combination thereof. The data query system 100 can also be a software system. When the data query system 100 is a software system, the data query system 100 can be deployed on a single computing device or on a computing device cluster composed of multiple computing devices. The above computing devices can be terminal servers or computing devices in a data center (such as servers, virtual machines, containers, etc.). The client 200 can be software or an application deployed on a terminal device. For example, it can be a web client. Among them, the above terminal devices include, for example, desktop computers, laptop computers, tablet computers, smart phones, wearable devices, etc. In addition, the data query system 100 and the client 200 can be deployed on the same device or on different devices respectively.

[0037] Next, in combination with Figure 2 the schematic flowchart of the data query method shown, the process of the data query system 100 implementing data query will be described in detail.

[0038] Step 101: The client 200 sends a data query request to the data query system 100. Correspondingly, the data query system 100 receives the data query request sent by the client 200.

[0039] Among them, the data query request is used to obtain the data required by the user from the data source 101. The data query request includes the identifier of the data model, query fields, and a first filter. The identifier of the data model is used to specify the data model from which to query the data required by the user, and specifically, it can be information such as the name or number of the data model that can indicate the data model. The query fields are used to specify the fields from which to query the data required by the user. The query fields are fields in the data model, and specifically, they can include one or more of the fields in the data tables included in the data model, the calculation fields defined in the data model, and the type conversion fields defined in the data model. The first filter is used to define the query conditions and indicate the data required by the user to be filtered out from the data corresponding to the query fields. The first filter can include multiple filter fields, the values of each filter field, and the connection fields between the multiple filter fields. The multiple filter fields all belong to the query fields. The value of the filter field refers to the specific value or range of the filter field. For example, if the filter field is a field of time type, the value of the filter field can be a specific date. The connection fields between the multiple filter fields are used to indicate the connection relationship between the multiple filter fields. For example, two filter fields can be connected by an "and" field to indicate an "and" relationship between the two fields. Another example is that two filter fields can also be connected by an "or" field to indicate an "or" relationship between the two fields. The connection fields between the multiple filter fields can be configured by the user or by the data query system 100.

[0040] Specifically, the user logs in to the data query system 100 through the client 200 based on the account. After successful login, the data query system 100 displays a query interface to the user through the client 200. The user can operate on this query interface through the client 200. For example, the user can specify the data model and query fields and configure the first filter. Accordingly, the client 200 generates a data query request according to the user's operation. The data query request can be a request in JSON format or a logical SQL statement (i.e., an SQL statement used to describe the logical structure of the data query operation expected by the user). Then, the client 200 sends the generated data query request to the data query system 100.

[0041] Please refer to Figure 3 , Figure 3 which exemplarily shows a query interface that the data query system 100 may display to the user. As Figure 3As shown, the query interface 300 may include a data model selection bar 301, a query field selection bar 302, a filter configuration bar 303, and a data display area 304. Among them, the data model selection bar 301 is used to display the identifiers of multiple data models to the user, so as to indicate the user to select the data model to be queried. The query field selection bar 302 is used to display the fields in the data model to the user. For example, the fields in each data table included in the data model, the calculated fields in the data model, and the type conversion fields are used to indicate the user to select the query fields. The filter configuration bar 303 is used to indicate the user to configure the filter. The data display area 304 is used to display the queried data to the user. In a specific implementation, the user can select the identifier of a suitable data model (displayed as "Data Model 1" in the figure) from the identifiers of multiple data models displayed in the data model selection bar 301 according to actual needs. When the user selects "Data Model 1", the query field selection bar 302 will correspondingly display the fields in Data Model 1, which are displayed as the "Sales Amount" field, "Sales Region" field, "Sales Time" field in the sales fact table, the "product_ID" field, "product_name" field in the product dimension table, the "period / year" field, "period / month" field in the accounting period dimension table, and the "Product Type" field and "Accounting Period" field in the figure. The user can select query fields from the fields displayed in the query field selection bar 302. The query fields displayed in the figure include the "Sales Amount" field, "product_ID" field, "product_name" field, "period / year" field, "period / month" field, and the "Product Type" field and "Accounting Period" field. The user can also configure the filter in the filter configuration bar 303. The filter displayed in the figure is "Accounting Period Dimension Table". "Accounting Period" = 'Preset Time' and "Product Dimension Table". "Product Type" = 'Preset Type'. This filter is used to filter out data that meets the conditions that the accounting period is the preset time and the product type is the preset type. After that, the user clicks the "Query" button, causing the client 200 to generate a data query request and send the data query request to the data query system 100. Among them, the data query request includes the identifier of Data Model 1, the query fields selected by the user, and the filter configured by the user.

[0042] It should be understood that the above embodiments only exemplarily describe an implementation manner of the client 200 sending a data query request to the data query system 100. In actual applications, the client 200 may also send a data query request to the data query system 100 in other ways, and this embodiment does not limit this.

[0043] Step 102: The data query system 100 obtains the configuration information of the data model according to the data query request.

[0044] Specifically, the data query system 100 pre-stores the identifiers of multiple preset data models and the configuration information of the preset data models indicated by each identifier. The identifier of the preset data model can be information such as the name or number of the preset data model that can be used to indicate the preset data model. For the relevant description of the configuration information of the preset data model, refer to the previous text. The data query system 100 can obtain the identifier of the data model by parsing the data query request, and then obtain the configuration information of the data model according to the matching relationship between the identifier of the data model and the identifiers of multiple preset data models. Among them, the configuration information of the data model is the configuration information of the preset data model indicated by the matching identifier. It should be understood that in practical applications, the data query system 100 can use existing algorithms in the industry that have better effects on string matching (for example, cosine similarity algorithm, word embedding algorithm, etc.) to determine the matching relationship between the identifier of the data model and the identifiers of multiple preset data models. For the sake of simplicity, the specific matching process is not described in detail here.

[0045] The configuration information of the data model includes the association relationships between multiple data tables. The multiple data tables include a first data table and a second data table. Both the first data table and the second data table can be fact tables or dimension tables. Taking the first data table and the second data table as an example, the association relationship between any two of the multiple data tables is described: The association relationship between the first data table and the second data table includes the identifier of the first data table, the identifier of the second data table, and the association field. The first data table and the second data table are associated through the association field. Among them, the association field includes field M in the first data table and field N in the second data table, and field M is associated with field N.

[0046] The configuration information of the data model further includes the fields defined in the data model (hereinafter referred to as "configuration fields"), and the number of configuration fields can be one or more, which is not limited in this embodiment. The configuration fields in the data model can include calculation fields and / or type conversion fields. Among them, a calculation field refers to a field obtained by performing arithmetic operations on the fields of a data table, for example, a field obtained by performing arithmetic operations, logical processing, concatenation processing, etc. on the fields of a data table. A calculation field can include an identifier of the calculation field and a calculation expression. The identifier of the calculation field can specifically be information such as the ID of the calculation field that can indicate the field. The calculation expression is used to indicate how to perform arithmetic operations on the fields of the data table to obtain the calculation field, that is, according to the calculation expression, the fields in the data table can be processed into a calculation field. A type conversion field refers to a field obtained by performing type conversion processing on the type of a field in a data table. A type conversion field can include an identifier of the type conversion field and a conversion expression. The identifier of the type conversion field can specifically be information such as the ID of the type conversion field that can indicate the field. The conversion expression is used to indicate performing type conversion processing on the fields in the data table to obtain the type conversion field, that is, according to the conversion expression, the fields in the data table can be processed into a type conversion field.

[0047] Take Figure 3 Data model 1 in Figure 4As shown in the figure, the data model 1 includes a sales fact table, a product dimension table, and an accounting period dimension table. Among them, the sales fact table includes a "period" field, a "product_ID" field, a "sales amount" field, a "sales region" field, and a "sales time" field. The product dimension table includes a "product_ID" field, a "product_name" field, and a "product_code" field. The accounting period dimension table includes a "period" field, a "period / year" field, and a "period / month" field. The data model 1 also defines a calculated field and a type conversion field. The configuration information of the data model 1 includes the association relationship between the sales fact table and the product dimension table, the association relationship between the sales fact table and the accounting period dimension table, the calculated field, and the type conversion field. Among them, the association relationship between the sales fact table and the product dimension table includes the identifier of the sales fact table (shown as "sales fact table" in the figure), the identifier of the product dimension table (shown as "product dimension table" in the figure), and the associated field (i.e., the "product_ID" field in the sales fact table and the "product_ID" field in the product dimension table). The association relationship between the sales fact table and the accounting period dimension table includes the identifier of the sales fact table (shown as "sales fact table" in the figure), the identifier of the accounting period dimension table (shown as "accounting period dimension table" in the figure), and the associated field (i.e., the "period" field in the sales fact table and the "period" field in the product dimension table). The calculated field includes the identifier of the calculated field and the calculation expression. In the figure, the identifier of the calculated field is "product dimension table". "product type", and the calculation expression is concat("product dimension table". "product_code", 'type'). This calculation expression is used to indicate that appending the string "type" after the "product_code" field in the product dimension table can obtain the "product dimension table". "product type" field. The type replacement field includes the identifier of the type replacement field and the conversion expression. In the figure, the identifier of the type conversion field is "accounting period dimension table". "accounting period", and the conversion expression is cast("accounting period dimension table". "period" AS VARCHAR). This conversion expression is used to indicate that converting the type of the "accounting period" field in the accounting period dimension table from a numeric type to a string type can obtain the "accounting period dimension table". "accounting period" field.

[0048] Step 103: The data query system 100 performs a splitting operation on the first filter to obtain one or more second filters.

[0049] Specifically, the data query system 100 obtains a first filter by parsing a data query request, and then performs lexical analysis and syntax analysis on the first filter to obtain connection fields in the first filter. The number of connection fields in the first filter can be one or more, and the connection fields in the first filter are used to represent the connection relationship between different fields in the filter. After that, the data query system 100 performs a splitting operation on the first filter according to the connection fields in the first filter to obtain one or more second filters. It should be understood that when only one second filter is obtained after the splitting operation, it means that this second filter is the first filter.

[0050] In some embodiments, the connection fields in the first filter include "and" fields, and the "and" fields are used to indicate that the fields connected by this field have a parallel relationship. The data query system 100 can perform lexical analysis and syntax analysis on the first filter according to algorithms that are already available in the industry and have better effects on lexical analysis and syntax analysis of expressions (for example, regular expressions and recursive descent analysis algorithms) to identify the "and" fields in the first filter. After that, the data query system 100 splits the first filter into multiple second filters according to the identified "and" fields. Each second filter does not include an "and" field, and the two fields connected by the "and" field are respectively fields in different second filters. It should be understood that compared with other connection fields in the first filter (for example, "or" fields), the multiple second filters obtained by splitting the first filter according to the "and" fields are more convenient for filter optimization operations and are easier to query the inner data tables of the data source 101.

[0051] Take Figure 3 the filter in (i.e., "Accounting Period Dimension Table". "Accounting Period" = 'Preset Time' and "Product Dimension Table". "Product Type" = 'Preset Type') as an example to illustrate the splitting operation of the filter: The data query system 100 can determine the "and" field in this filter by performing lexical analysis and syntax analysis on this filter, and then split the filter into two filters according to the "and" field. These two filters are respectively: "Accounting Period Dimension Table". "Accounting Period" = "Preset Time", "Product Dimension Table". "Product Type" = 'Preset Type'.

[0052] Step 104, the data query system 100 performs an optimization operation on the one or more second filters obtained by splitting according to the configuration information of the data model to obtain one or more new filters.

[0053] Taking one second filter as an example, the data query system 100 can optimize this filter in any of the following ways:

[0054] Method 1: The data query system 100 optimizes the second filter according to the matching relationship between the first field in the second filter and the configuration fields in the data model to obtain a new filter.

[0055] Specifically, the data query system 100 obtains the configuration fields in the data model according to the configuration information of the data model, then determines the matching relationship between the first field and the configuration fields in the data model, and then obtains the source fields on which the first field depends according to the matching relationship between the first field and the configuration fields in the data model. The source fields are the fields in the first data table in the data model. After that, the data query system 100 optimizes the second filter according to the above source fields to obtain a new filter.

[0056] In some embodiments, taking the first field and a configuration field (for example, configuration field A) in the data model as an example, the data query system 100 can determine the matching relationship between the two in the following way: The data query system 100 obtains the identifier of the first field and the identifier of configuration field A, and then uses a string matching algorithm to determine whether the identifier of the first field is the same as the identifier of configuration field A. When the identifier of the first field is the same as the identifier of configuration field A, it indicates that the first field matches configuration field A; when the identifier of the first field is different from the identifier of configuration field A, it indicates that the first field does not match configuration field A. Among them, the above string matching algorithm can use existing algorithms in the industry that have better effects on string matching, such as the cosine similarity algorithm, the word embedding algorithm, etc.

[0057] In some embodiments, the configuration fields in the data model include calculation fields and type conversion fields.

[0058] When the first field matches a calculated field in the data model, the data query system 100 can obtain the source field on which the first field depends in the following manner: The data query system 100 obtains the calculation expression corresponding to the matching calculated field according to the configuration information of the data model, and then judges whether the calculation expression meets the preset conditions by performing lexical analysis and syntactic analysis on the calculation expression. When the calculation expression meets the preset conditions, the data query system 100 processes the first field according to the calculation expression to obtain the source field on which the first field depends. Among them, the preset condition is that the calculation expression includes constants, and the operation relationship between the source field and the constants in the calculation expression is reversible. That is to say, for a calculation expression that meets the preset conditions, the data query system 100 can perform inverse operation processing on the first field and the constants according to the calculation expression to obtain the source field on which the first field depends. The data query system 100 can also process the value of the first field in the second filter according to the above calculation expression to obtain a new value, and then replace the first field in the second filter with the above source field and replace the value of the first field in the second filter with the new value, so as to obtain a new filter.

[0059] For example, in the example described in step 103 above, the original filter is "Product Dimension Table". "Product Type" = 'Preset Type'. From the configuration information of data model 1, it can be seen that the field in this filter (i.e., the "Product Type" field) is a calculated field, and the calculation expression is concat("Product Dimension Table". "product_code", 'Type'). Since this calculation expression includes a constant (i.e., 'Type'), and the operation relationship (i.e., the concat() function) between the source field (the "product_code" field in the Product Dimension Table) and the constant is reversible, therefore, this calculation expression meets the preset conditions. Processing the field in the original filter according to this calculation expression can obtain the corresponding source field, so as to obtain a new filter as "Product Dimension Table". "product_code" = 'Preset'.

[0060] When the first field matches a type conversion field in the data model, the data query system 100 can obtain the source field on which the first field depends in the following manner: The data query system 100 obtains the conversion expression corresponding to the matching type conversion field according to the configuration information of the data model, and then processes the first field according to the conversion expression to obtain the source field on which the first field depends. The data query system 100 can also process the value of the first field in the second filter according to the above conversion expression to obtain a new value, and then replace the first field in the second filter with the above source field and replace the value of the first field in the second filter with the new value, so as to obtain a new filter.

[0061] For example, in the example described in step 103 above, the original filter is "Accounting Period Dimension Table". "Accounting Period" = 'Preset Time', where the "Preset Time" in the original filter is a string type value. From the configuration information of data model 1, it can be known that the field in this filter (i.e., the "Accounting Period" field) is a type conversion field, and the conversion expression is cast("Accounting Period Dimension Table". "period" AS VARCHAR). Processing the field in the original filter according to this conversion expression can obtain the corresponding source field (i.e., the "period" field in the Accounting Period Dimension Table), so as to obtain the new filter as "Accounting Period Dimension Table". "period" = 'Preset Time', where the "Preset Time" in the new filter is a numeric type value.

[0062] It should be understood that when the field in the filter is a calculated field or a type conversion field, the field in the new filter obtained by optimizing the filter through Method 1 is the source field on which the calculated field or the type conversion field depends. Therefore, the new filter can be directly used to filter the data table. That is to say, compared with the filter before optimization, the new filter can improve the efficiency of data query.

[0063] Method 2: The data query system 100 optimizes the second filter according to the association relationship between the first field in the second filter and the data tables in the data model to obtain a new filter.

[0064] Specifically, the data query system 100 obtains a second field, where the second field is the first field or the source field on which the first field depends. The data query system 100 also obtains the association relationship between the data tables in the data model according to the configuration information of the data model, and then obtains the field associated with the second field according to the association relationship between the second field and the data tables in the data model, and then replaces the first field in the second filter with the field associated with the second field to obtain a new filter.

[0065] In some embodiments, the data query system 100 can obtain the second field in the following manner: The data query system 100 obtains the configuration fields in the data model according to the configuration information of the data model, and then determines the matching relationship between the first field and the configuration fields in the data model. When the first field does not match any of the configuration fields in the data model, it is determined that the second field is the first field. When the first field matches any one of the configuration fields in the data model, it is determined that the second field is the source field on which the first field depends. It should be understood that the specific implementation process of determining the matching relationship between the first field and the configuration fields in the data model and obtaining the source field on which the first field depends in this step can refer to the relevant description of Method 1 above. For the sake of simplicity, it will not be repeated here.

[0066] For example, still taking the example described in step 103 above, the original filter is "Accounting Period Dimension Table". "Accounting Period" = 'preset time'. From the configuration information of data model 1, it can be known that the field in this filter (i.e., the "Accounting Period" field) is a type conversion field, and the source field on which this field depends is the "period" field in the Accounting Period Dimension Table. The field associated with the above source field is the "period" field in the Sales Fact Table. Therefore, the new filter is "Sales Fact Table". "period" = "preset time".

[0067] It should be understood that the new filter obtained by method two can directly filter most of the data in the data table according to the hidden field, so that the amount of data during the subsequent association operation between this data table and other data tables is greatly reduced, thereby improving the efficiency of data query.

[0068] Optionally, the above method one and method two can be executed simultaneously or successively. Moreover, when the above method one and method two are executed successively, the execution order of the two can be not limited.

[0069] Step 105: The data query system 100 performs a data query operation on the data model according to the obtained one or more new filters, and sends the queried data to the client 200. Correspondingly, the client 200 receives the data sent by the data query system 100.

[0070] Specifically, the data query system 100 generates an SQL statement according to the query field, one or more new filters, and the configuration information of the data model, and then performs a data query operation on the data model according to the SQL statement to obtain the data required by the user. After that, the data query system 100 sends the queried data to the client 200.

[0071] Step 106: The client 200 displays the queried data.

[0072] In the data query method described in the above steps 101 to 106, after the data query system 100 obtains the filter configured by the user, it first optimizes the filter. The new filter obtained by the optimization can directly filter the corresponding data table according to the fields in the data table. Therefore, using the new filter to perform a data query operation on the data model can improve the efficiency of data query.

[0073] In the above text, in combination with Figures 1 to 4 , the data query method provided by the present application is described in detail. Next, in combination with Figure 5 , the data query system 100 that executes the above data query method will be described.

[0074] Figure 5Exemplarily shows a schematic structural diagram of the data query system 100. It should be understood that Figure 5 It only exemplarily shows a way of dividing the structure of the data query system 100. In actual applications, there may be other ways of dividing the structure of the data query system 100, which are not specifically limited in this embodiment. For example Figure 5 As shown, the data query system 100 includes an acquisition module 101, a filter optimization module 102, and a data query module 103. Among them, the acquisition module 101, the filter optimization module 102, and the data query module 103 work together to implement the steps performed by the data query system 100 in the above data query method. Specifically, the acquisition module 101 is used to perform the relevant steps of receiving the data query request sent by the client 200 in the above step 101. The filter optimization module 102 is used to perform the above steps 102 to 104. The data query module 103 is used to perform the above step 105.

[0075] This application also provides a computing device. The computing device can be a computing device in a terminal server or a data center. Figure 6 Exemplarily shows a schematic structural diagram of the computing device provided by this application. For example Figure 6 As shown, the computing device 400 includes a bus 401, a processor 402, a memory 403, and a communication interface 404, and the processor 402, the memory 403, and the communication interface 404 communicate with each other through the bus 401.

[0076] The bus 401 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation Figure 6 only one line is shown in, but this does not mean that the computing device 400 has only one bus or one type of bus. The bus 401 can include a path for transmitting information between various components (for example, the processor 402, the memory 403, and the communication interface 404) of the computing device 400.

[0077] The processor 402 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), a data processing unit (DPU), a system on chip (SoC), an offload card, an acceleration card, and other computing units with computing capabilities.

[0078] The memory 403 may include a volatile memory, such as a random access memory (RAM). The memory 403 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0079] Executable code is stored in the memory 403. The processor 402 executes the code stored in the memory 403 to implement the functions of the above-mentioned acquisition module 101, filter optimization module 102, and data query module 103, respectively. That is to say, instructions for executing the data query system 100 in the above data query method are stored on the memory 403.

[0080] The communication interface 404 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 400 and other devices or a communication network. For example, the computing device 400 communicates with the client 200 through the communication interface 404.

[0081] It should be understood that the computing device 400 provided in this application may correspond to the data query system 100 in this application, and the above and other operations and / or functions of each module in the computing device 400 are respectively for implementing Figure 2 the corresponding processes of each method in, and for the sake of brevity, they will not be elaborated here.

[0082] This application also provides a computing device cluster. The computing device cluster includes a plurality of computing devices, and the plurality of computing devices in the computing device cluster may include one or more of the computing devices in a terminal server or a data center. Figure 7 Exemplarily shows a schematic structural diagram of the computing device cluster provided in this application. AsFigure 7 As shown in Figure 7 , the computing device cluster 500 includes a plurality of computing devices 400, where the plurality of computing devices 400 can be connected through a network (such as a wide area network or a local area network, etc.).

[0083] In some embodiments, the instructions stored in the memory 403 of each computing device 400 in the computing device cluster 500 can implement the functions of one or more of the above-mentioned acquisition module 101, filter optimization module 102, and data query module 103. That is to say, the memory 403 of each computing device 400 in the computing device cluster 500 can respectively store some instructions executed by the data query system 100 in the above-mentioned data query method, so that the computing device cluster 500 executes the instructions executed by the data query system 100 in the above-mentioned data query method. In a specific implementation, different instructions can be stored in the memories 403 of different computing devices 400 in the computing device cluster 500, or the same instructions can also be stored in the memories 403 of some computing devices 400 in the computing device cluster 500. For example, some computing devices 400 all store instructions for implementing the function of the filter optimization module 102.

[0084] It should be understood that the computing device cluster 500 provided according to the present application can correspond to the data query system 100 in the present application, and the above and other operations and / or functions of each module in the computing device cluster 500 are respectively for implementing Figure 2 the corresponding processes of each method in Figure 2 . For the sake of brevity, they will not be described in detail here.

[0085] The present application also provides a computer program product containing instructions. The computer program product can be software or a program product containing instructions that can run on one or more computing devices or be stored in any available medium. When the computer program product runs on a single computing device or a computing device cluster, it causes the single computing device or the computing device cluster to execute the data query method described above.

[0086] The above content can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in the embodiments of the present application according to the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk (SSD).

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data query method, applied to a data query system, characterized in that, The method includes: Receiving a data query request, where the data query request includes an identifier of a data model and a first filter; Obtaining configuration information of the data model according to the identifier of the data model, where the configuration information of the data model includes the association relationship between data tables in the data model or the fields defined in the data model; Optimizing the first filter according to the configuration information of the data model to obtain a new filter; Performing a data query operation on the data model according to the new filter.

2. The method according to claim 1, characterized in that, The optimizing the first filter according to the configuration information of the data model to obtain a new filter includes: Splitting the first filter according to the connection relationship between the fields in the first filter to obtain a second filter; Optimizing the second filter according to the configuration information of the data model to obtain the new filter.

3. The method according to claim 2, characterized in that, The second filter includes a first field, and the configuration information of the data model includes the association relationship between data tables in the data model. The optimizing the second filter according to the configuration information of the data model to obtain the new filter includes: Obtaining a second field, where the second field is the first field or the source field on which the first field depends, and the source field is a field in a data table in the data model; Obtaining the fields associated with the second field according to the second field and the association relationship between data tables in the data model; Replacing the first field in the second filter with the fields associated with the second field to obtain the new filter.

4. The method according to claim 2, characterized in that, The second filter includes a first field, and the configuration information of the data model includes the fields defined in the data model. The optimizing the second filter according to the configuration information of the data model to obtain a new filter includes: Obtaining the source field on which the first field depends according to the matching relationship between the first field and the fields defined in the data model, where the source field is a field in a data table in the data model; Replacing the first field in the second filter with the source field to obtain the new filter.

5. The method according to claim 4, characterized in that, The fields defined in the data model include calculation fields. The obtaining the source field on which the first field depends according to the matching relationship between the first field and the fields defined in the data model includes: Obtaining a calculation expression corresponding to the matching calculation field according to the matching relationship between the first field and the calculation fields in the data model, where the calculation expression is used to indicate the relationship between the matching calculation field and the source field; Processing the first field according to the calculation expression to obtain the source field.

6. The method according to claim 4, characterized in that, The fields defined in the data model include type conversion fields. The obtaining the source field on which the first field depends according to the matching relationship between the first field and the fields defined in the data model includes: Obtaining a conversion expression corresponding to the matching type conversion field according to the matching relationship between the first field and the type conversion fields in the data model, where the conversion expression is used to indicate the relationship between the matching type conversion field and the source field; Process the first field according to the conversion expression to obtain the source field.

7. A data query system, characterized in that, The system includes: An acquisition module, configured to receive a data query request, where the data query request includes an identifier of a data model and a first filter; A filter optimization module, configured to obtain configuration information of the data model according to the identifier of the data model, where the configuration information of the data model includes an association relationship between data tables in the data model or fields defined in the data model; optimize the first filter according to the configuration information of the data model to obtain a new filter; A data query module, configured to perform a data query operation on the data model according to the new filter.

8. The system according to claim 7, characterized in that, The filter optimization module is configured to split the first filter according to a connection relationship between fields in the first filter to obtain a second filter; optimize the second filter according to the configuration information of the data model to obtain the new filter.

9. The system according to claim 8, characterized in that, The second filter includes a first field, and the configuration information of the data model includes an association relationship between data tables in the data model. The filter optimization module is configured to obtain a second field, where the second field is the first field or a source field on which the first field depends, and the source field is a field in a data table in the data model; obtain fields associated with the second field according to the second field and the association relationship between data tables in the data model; replace the first field in the second filter with the fields associated with the second field to obtain the new filter.

10. The system according to claim 8, characterized in that, The second filter includes a first field, and the configuration information of the data model includes fields defined in the data model. The filter optimization module is configured to obtain a source field on which the first field depends according to a matching relationship between the first field and the fields defined in the data model, where the source field is a field in a data table in the data model; replace the first field in the second filter with the source field to obtain the new filter.

11. The system according to claim 10, wherein The fields defined in the data model include calculation fields. The filter optimization module is configured to obtain a calculation expression corresponding to a matching calculation field according to a matching relationship between the first field and the calculation fields in the data model, where the calculation expression is used to indicate a relationship between the matching calculation field and the source field; Process the first field according to the calculation expression to obtain the source field.

12. The system according to claim 10, wherein The fields defined in the data model include type conversion fields. The filter optimization module is configured to obtain a conversion expression corresponding to a matching type conversion field according to a matching relationship between the first field and the type conversion fields in the data model, where the conversion expression is used to indicate a relationship between the matching type conversion field and the source field; Process the first field according to the conversion expression to obtain the source field.

13. A computing device, wherein It includes a processor and a memory, and the processor is configured to execute instructions stored in the memory to enable the computing device to execute the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, wherein Comprising computer program instructions which, when executed by a computing device, cause the computing device to perform the method according to any one of claims 1 to 6.

Images