Data query method and apparatus

Abstract

Embodiments of the present specification provide a data query method and device, the method comprising: receiving a data query request for a target data table, the data query request including filter conditions for a plurality of fields of the target data table, the target data table supporting a plurality of reading methods, each of the plurality of reading methods using part of the fields of the target data table for filter query; in response to the presence of logical AND and logical OR operators in the filter conditions, converting the filter conditions into an equivalent disjunctive normal form; and generating a data query plan based on the disjunctive normal form and the plurality of reading methods.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of computer technology, and in particular relate to data query methods and apparatus. Background Technology

[0002] In many data query scenarios, data providers typically do not directly expose data tables to data users, nor do they allow users to directly query data tables using SQL (Structured Query Language) statements, in order to ensure data privacy and security. In such cases, data providers usually impose some query restrictions on the data source, such as using API (Application Programming Interface) queries to allow queries on only a specific field or a set of fields, thereby limiting the scope of the user's query. Summary of the Invention

[0003] The purpose of this invention is to provide a data query scheme that can rewrite complex filtering conditions in scenarios where data source queries are limited, and automatically generate feasible data query plans based on the reading methods supported by the data source, thereby reducing the additional costs and potential risks of manually designing query plans.

[0004] This specification provides a data query method, comprising: receiving a data query request for a target data table, the data query request including filtering conditions for multiple fields of the target data table, the target data table supporting several reading methods, each of the several reading methods using a subset of fields of the target data table for filtering; in response to the presence of logical AND and logical OR operators in the filtering conditions, converting the filtering conditions into an equivalent disjunctive normal form; and generating a data query plan based on the disjunctive normal form and the several reading methods.

[0005] A second aspect of this specification provides a data query apparatus, comprising: a receiving unit configured to receive a data query request for a target data table, the data query request including filtering conditions for multiple fields of the target data table, the target data table supporting several reading methods, each of the several reading methods using a subset of fields of the target data table for filtering; a conversion unit configured to convert the filtering conditions into an equivalent disjunctive normal form in response to the presence of logical AND and logical OR operators in the filtering conditions; and a generation unit configured to generate a data query plan based on the disjunctive normal form and the several reading methods.

[0006] A third aspect of this specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in any implementation of the first aspect.

[0007] A fourth aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements a method as described in any implementation of the first aspect.

[0008] The fifth aspect of this specification provides a computer program that, when executed in a computer, causes the computer to perform the method described in any implementation of the first aspect.

[0009] In the solution provided in the embodiments of this specification, the target data table, serving as the data source, provides a data query service with query restrictions. Specifically, it supports several read methods, all of which use certain fields of the target data table for filtering queries. In this solution, a data query request for the target data table can be received, including filtering conditions for multiple fields of the target data table. Subsequently, in response to the presence of logical AND and logical OR operators in the filtering conditions, the filtering conditions can be converted into an equivalent disjunctive normal form. Based on the disjunctive normal form and the several read methods, a data query plan is generated for querying the target data table. Therefore, complex filtering conditions in scenarios with restricted data source queries can be rewritten, and feasible data query plans can be automatically generated according to the read methods supported by the data source, reducing the additional costs and potential risks of manually designing query plans. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating one application scenario in which the embodiments of this specification can be applied;

[0012] Figure 2 This is a flowchart of the data query method in the embodiments of this specification;

[0013] Figure 3 This is a schematic diagram of the initial query plan;

[0014] Figure 4 This is a flowchart of the initial query plan optimization process;

[0015] Figure 5 This is a schematic diagram of the updated initial query plan;

[0016] Figure 6 This is a schematic diagram of the updated initial query plan;

[0017] Figure 7 This is a schematic diagram of the data query device in the embodiments of this specification. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0019] As mentioned earlier, in many data query scenarios, data providers typically do not directly expose data tables to data users, nor do they allow users to directly query data tables using SQL statements, in order to ensure data privacy and security. In such cases, data providers usually impose some query restrictions on the data source, such as allowing queries on only a specific field or a set of fields via API queries, thereby limiting the scope of the user's query.

[0020] Currently, in scenarios where data source queries are limited, developers typically design query plans in advance. A query plan can be understood as the specific steps involved in executing the query. For different data usage scenarios, the query plan needs to be redesigned each time. This introduces more manual labor costs into the development process, and manual design also carries the risk of errors.

[0021] To reduce the additional costs and potential risks of manually designing query plans, this specification provides a data query scheme that can rewrite complex filtering conditions in scenarios where data source queries are limited, and automatically generate feasible data query plans based on the reading methods supported by the data source.

[0022] Figure 1 This is a schematic diagram illustrating one application scenario to which the embodiments of this specification can be applied. Figure 1 The application scenario shown may include user device 101 used by user A, database server 102, and data table table1 included in database server 102. Table1 includes multiple fields, such as... Figure 1The fields shown are col1, col2, col3, col4, col5, and col6. It should be noted that... Figure 1 The "##" shown can represent field values.

[0023] Additionally, tabel1 supports several read methods, all of which use a subset of tabel1's fields for filtering queries. For example, ... Figure 1 As shown, tabel1 supports the read methods read_method1(col2, col3) and read_method2(col4). read_method1 can perform multi-column filtering queries using fields col2 and col3, and its query effect is equivalent to the SQL statement "select * from tabel1 where col2 = ? and col3 = ?". read_method2 can perform filtering queries using field col4, and its query effect is equivalent to the SQL statement "select * from tabel1 where col4 = ?". It should be noted that the "?" in the SQL statement can be called a parameter placeholder; the position of this placeholder is used to place the actual field value.

[0024] When user A wants to query data in table1 that satisfies the following filter condition F1: col1>18 and (col2=123 and col3='AA' or (col4='D' and col5<8) or (col4='D' and col6>3)), user A can send a data query request including filter condition F1 to database server 102 through user device 101. Database server 102 can respond to the presence of logical AND and logical OR operators in filter condition F1 by converting filter condition F1 into an equivalent disjunctive normal form. As is well known, disjunction represents "OR", and conjunction represents "AND". A disjunction consisting of a finite number of simple conjunctions is called a disjunctive normal form. According to the definition of disjunctive normal form, filter condition F1 can be specifically converted into the following equivalent disjunctive normal form F2: (col1>18 and col2=123 and col3='AA') or (col1>18 and col4='D' and col5<8) or (col1>18 and col4='D' and col6>3). Subsequently, database server 102 can generate a data query plan based on disjunctive normal form F2 and the read methods supported by table1, for use in querying data on table1.

[0025] The data query scheme provided in the embodiments of this specification will be described below with reference to specific examples.

[0026] See Figure 2 This is a flowchart of the data query method in the embodiments of this specification. This method can be executed by any device, platform, or device cluster with data storage, computing, and processing capabilities; for example, it can be executed by a database server (such as...). Figure 1 The database server 102 shown, data flow hub platform, privacy collaboration application platform or data matrix service platform, etc. are executed.

[0027] like Figure 2 As shown, firstly, in step S201, a data query request for a target data table is received. The data query request includes filtering conditions for multiple fields of the target data table. The target data table supports several reading methods, and each of these reading methods uses some fields of the target data table for filtering and querying.

[0028] Specifically, it can receive data query requests from users targeting a specific data table. After receiving the query request, it can analyze the filtering conditions to determine if logical AND and OR operators are present. In practice, the presence of logical AND and OR operators in the filtering conditions indicates that the conditions are complex and require rewriting.

[0029] In step S203, in response to the presence of logical AND and logical OR operators in the filter condition, the filter condition is converted into an equivalent disjunctive normal form.

[0030] Specifically, any feasible transformation method can be used to convert the filtering condition into an equivalent disjunctive normal form.

[0031] In one example, the filter condition can be converted into a matrix expression, calculated, and the disjunctive normal form generated based on the result. Taking the filter condition F1 described earlier as an example, we can first determine the one-hot encodings corresponding to the fields col1, col2, col3, col4, col5, and col6. Assuming that table1 includes these 6 fields and the length of the one-hot encoding is 6, col1 can correspond to [1 0 0 0 0 0], col2 can correspond to [0 1 0 0 0 0], col3 can correspond to [0 0 1 0 0 0], col4 can correspond to [0 0 0 1 0 0], col5 can correspond to [0 0 0 0 1 0], and col6 can correspond to [0 0 0 0 0 1]. Here, the "1" in the one-hot encoding can represent the field in table1 that is at the same position as the "1". Taking the one-hot encoding [1 0 0 0 0 0] as an example, the "1" is in the first position from left to right, which can represent the first field in table1, i.e., field col1. Then, "*" can be used to represent "and", and "+" can be used to represent "or". Based on the determined one-hot encodings, the filtering condition F1 is transformed into the following matrix expression:

[0032] [1 0 0 0 0 0]*([0 1 0 0 0 0]*[0 0 1 0 0 0]+([0 0 0 1 0 0]*[00 0 0 10])+([0 0 0 1 0 0]*[0 0 0 0 0 1])).

[0033] Next, the above matrix-based expression can be calculated to obtain the following results:

[0034]

[0035] In the above calculation results, each row can represent a conjunctive expression in disjunctive normal form. The "1" in the row can represent that the conjunctive expression includes a filtering condition on the field in table1 that is in the same position as the "1". Taking the first row "111000" in the above calculation results as an example, this row can represent a conjunctive expression in disjunctive normal form. In order from left to right, the first "1" in the row means that the conjunctive expression includes a filtering condition on the first field col1 in table1 (i.e., col1>18), the second "1" in the row means that the conjunctive expression includes a filtering condition on the second field col2 in table1 (i.e., col2=123), and the third "1" in the row means that the conjunctive expression includes a filtering condition on the third field col3 in table1 (i.e., col3='AA').

[0036] Then, based on the above calculation results, the following equivalent disjunctive normal form F2 can be generated:

[0037] (col1>18and col2=123and col3='AA')or(col1>18and col4='D'and col5<8)or(col1>18and col4='D'and col6>3).

[0038] In another example, when converting the filtering condition into an equivalent disjunctive normal form, one can first use the disjunctive law to obtain a disjunction, and then use the absorptivity and idempotency laws to simplify it, thus obtaining an equivalent disjunctive normal form.

[0039] In step S205, a data query plan is generated based on the disjunctive paradigm and several reading methods.

[0040] Specifically, a data query plan can be generated based on the disjunction paradigm obtained in step S203 and several reading methods supported by the target data table.

[0041] In practice, this disjunctive normal form consists of multiple conjunctive expressions. In one example, when all the conjunctive expressions are sub-conditions allowed by the aforementioned read methods, for each of the multiple conjunctive expressions, a query invocation plan corresponding to that conjunctive expression can be generated based on that conjunctive expression and the read methods it satisfies. Then, a result merging plan can be generated, and this result merging plan and the individual query invocation plans can be combined to form a data query plan. The result merging plan serves as the next execution plan for each query invocation plan within the data query plan. It should be noted that the query invocation plan may include, for example, a query invocation operator (e.g., the QueryCall operator), the read methods used by that query invocation operator, and the values ​​in the conjunctive expressions corresponding to that query invocation plan. These values ​​act as input parameters for the read methods. The result merging plan may include a union operator (e.g., the Union operator).

[0042] In another example, when the disjunctive normal form obtained after the transformation contains subconditions that are not allowed by the aforementioned read methods, the first subconditions that are allowed by the aforementioned read methods can be extracted from the disjunctive normal form. Then, for each of these first subconditions, a query invocation plan can be generated based on the first subcondition and the read method it satisfies. Finally, a data query plan can be generated based on the generated query invocation plans.

[0043] Specifically, when all the aforementioned conjunctions include the first sub-conditions of each of the aforementioned first sub-conditions, and some conjunctions include second sub-conditions that are not allowed by the aforementioned reading methods, for each of the aforementioned partial conjunctions, a first result filtering plan including the second sub-condition of the conjunction can be generated. This first result filtering plan and the query invocation plan associated with the first sub-condition of the conjunction are then combined to form a first query plan, wherein the first result filtering plan serves as the next execution plan of the query invocation plan within the first query plan. Additionally, the first result filtering plan may also include a filtering operator (such as a Filter operator). For the remaining conjunctions, the query invocation plan associated with the first sub-condition of the conjunction can be used as the third query plan corresponding to that conjunction. Then, a result merging plan can be generated, and this result merging plan, each of the first query plans, and each of the third query plans are combined to form an initial query plan, wherein the result merging plan serves as the next execution plan of each of the first and third query plans within the initial query plan. Finally, a data query plan can be generated based on the initial query plan. For example, this initial query plan can be used as a data query plan.

[0044] When all of the aforementioned conjunctions include the first sub-conditions of each of the aforementioned first sub-conditions and the second sub-conditions that are not allowed by the aforementioned reading methods, for each of the aforementioned conjunctions, a first result filtering plan including the second sub-condition of the conjunction can be generated. This first result filtering plan and the query invocation plan associated with the first sub-condition of the conjunction are then combined to form a first query plan, wherein the first result filtering plan serves as the next execution plan of the query invocation plan within the first query plan. Next, a result merging plan can be generated, and this result merging plan and each of the first query plans are combined to form an initial query plan, wherein the result merging plan serves as the next execution plan of each of the first query plans within the initial query plan. It can be understood that the result merging plan specifically serves as the next execution plan of each of the first result filtering plans within the initial query plan. Then, a data query plan can be generated based on the initial query plan. For example, the initial query plan can be used as the data query plan.

[0045] Taking the target data table as table1 described above and the filter condition in the data query request as filter condition F1 described above as an example, combined with... Figure 3 This section describes the process of generating the initial query plan when all of the above conjunctive expressions include a first subcondition and a second subcondition. Specifically, Figure 3 This is a schematic diagram of the initial query plan. Figure 3 The arrows in the diagram illustrate the execution order of the various plans in the initial query plan.

[0046] Specifically, after transforming the filter condition F1 into the disjunctive normal form F2 described above, the first subcondition allowed by the read method read_method1(col2, col3) col2 = 123 and col3 = 'AA', and the first subcondition allowed by the read method read_method2(col4) col4 = 'D', can be extracted from the disjunctive normal form F2. For the first subcondition col2 = 123 and col3 = 'AA', based on this first subcondition and the read method it satisfies, the following can be generated: Figure 3 The query call plan pointed to by reference 301 is QueryCall(ReadMethod = read_method1, Param = 123, 'AA'). Here, "ReadMethod" represents the read method, and "Param" represents the input parameters of the read method. This query call plan indicates that, based on the read method read_method1 with input parameters 123 and 'AA', it retrieves data that satisfies col2 = 123 and col3 = 'AA'. For the first sub-condition col4 = 'D', based on this first sub-condition and the read method it satisfies, a query can be generated as follows: Figure 3 The query call plan pointed to by reference 302 is QueryCall(ReadMethod = read_method2, Param = 'D'). This query call plan indicates that a query is executed based on the read_method2 method, with the input parameter 'D', to retrieve data that satisfies col4 = 'D'.

[0047] For a single-table query, the filtering conditions filter1 and filter2 can be viewed as taking the filtering result result1 that satisfies filter1, and then filtering result1 to get the filtering result result2 that satisfies filter2.

[0048] Based on this, for the conjunctive expression (col1>18 and col2=123 and col3='AA') in disjunctive normal form F2, a first result filtering plan can be generated that includes the second subcondition col1>18 in the conjunctive expression. This first result filtering plan and the query invocation plan associated with the first subcondition col2=123 and col3='AA' in the conjunctive expression are then combined to form a first query plan. This first result filtering plan can be as follows: Figure 3The filter (Condition = col1 > 18) pointed to by reference 303 serves as the next execution plan in this first query plan. Here, "Condition" can represent a filtering condition. This first query plan can mean first retrieving data that satisfies col2 = 123 and col3 = 'AA', and then filtering out data from the query results that satisfies col1 > 18.

[0049] For the conjunctive expression (col1>18 and col4='D' and col5<8) in disjunctive normal form F2, a first result filtering plan can be generated that includes the second subcondition col1>18 and col5<8 in the conjunctive expression. This first result filtering plan and the query invocation plan associated with the first subcondition col4='D' in the conjunctive expression are then combined to form a first query plan. This first result filtering plan can be as follows: Figure 3 The filter (Condition = col1>18 and col5<8) pointed to by index 304 serves as the next execution plan in this first query plan. This first query plan can be interpreted as first retrieving data that satisfies col4 = 'D', and then filtering out data from the query results that satisfies col1>18 and col5<8.

[0050] For the conjunctive expression (col1>18 and col4='D' and col6>3) in disjunctive normal form F2, a first result filtering plan can be generated that includes the second subcondition col1>18 and col6>3 in the conjunctive expression. This first result filtering plan and the query invocation plan associated with the first subcondition col4='D' in the conjunctive expression are then combined to form a first query plan. This first result filtering plan can be as follows: Figure 3 The filter (Condition = col1>18 and col6>3) pointed to by index 305 serves as the next execution plan in this first query plan. This first query plan can be interpreted as first retrieving data that satisfies col4 = 'D', and then filtering out data from the query results that satisfies col1>18 and col6>3.

[0051] After constructing the first query plan corresponding to the three conjunctions in disjunctive normal form F2, the following can be generated: Figure 3The result merging plan pointed to by reference numeral 306 is called Union, and this result merging plan, along with the first query plans corresponding to the three conjunctive expressions, are combined to form the initial query plan. The result merging plan serves as the next execution plan for each of the first query plans within this initial query plan. This initial query plan can be represented as taking the union of the filtered query results for col1>18and col2=123and col3='AA', col1>18and col4='D'and col5<8, and col1>18and col4='D'and col6>3.

[0052] In one implementation, after executing step S205, step S207 may be executed next. Specifically, in step S207, a data query may be performed on the target data table based on the data query plan.

[0053] Figure 2 The corresponding implementation provides a solution that can rewrite complex filtering conditions in scenarios where data source queries are limited, and automatically generate feasible data query plans based on the reading methods supported by the data source. This reduces the additional costs and potential risks of manually designing query plans, and provides a simpler automated reading method for data reading in application development.

[0054] In practice, the second sub-conditions in the multiple conjunctions mentioned above may include the same third sub-condition. If the initial query plan is directly used as the data query plan when the second sub-conditions in these multiple conjunctions include the same third sub-condition, multiple query result filters will be executed based on this same third sub-condition during the data query process, which will affect the efficiency of the data query.

[0055] To effectively improve data query efficiency, the same filtering conditions can be extracted from each first result filtering plan, so that the filtering operation based on the same filtering conditions is executed in the next step of the Union operator.

[0056] Specifically, see Figure 4 It is a flowchart of the initial query plan optimization process.

[0057] like Figure 4 As shown, firstly, in step S401, a second result filtering plan including a third sub-condition is generated.

[0058] In step S403, the second result filtering plan is added to the initial query plan as the next execution plan of the result merging plan.

[0059] In step S405, each first result filtering plan is updated in the initial query plan so that the updated first result filtering plan no longer contains the third sub-condition.

[0060] As an example, suppose the above conjunctive expressions include a first conjunctive, a second conjunctive, and a third conjunctive, where the third subcondition is the entirety of the second subcondition in the first conjunctive and a portion of the second subconditions in each of the second and third conjunctive expressions. When updating each first result filtering plan in the initial query plan, the first result filtering plan can be removed from the first query plan associated with the first conjunctive, and the third subcondition and a logical AND operator used to connect the third subcondition can be removed from the first result filtering plans of the first query plans associated with the second and third conjunctive expressions. It should be noted that after removing the first result filtering plan from the first query plan associated with the first conjunctive, the next execution plan of the query call plan in that first query plan can be changed to a result merging plan.

[0061] In step S409, the updated initial query plan is used as the data query plan.

[0062] In one example, after updating each of the first result filtering plans in the initial query plan by executing step S405, the updated initial query plan can be used as the data query plan.

[0063] by Figure 3 Taking the initial query plan shown as an example, the optimization process of the initial query plan shown in steps S401-S405 will be illustrated. Figure 3 As shown in the diagram, labels 303, 304, and 305 respectively point to the first result filtering plan where the second sub-condition includes the same third sub-condition col1>18. A second result filtering plan that includes the third sub-condition col1>18 can be generated (e.g., ...). Figure 5 The filter (Condition = col1>18) pointed to by label 307 is added to the initial query plan as the next execution plan after the result merging plan pointed to by label 306. Furthermore, since the third sub-condition is the entirety of the second sub-condition in the first result filter plan pointed to by label 303 (i.e., the third sub-condition is identical to the second sub-condition), the first result filter plan pointed to by label 303 can be removed, thus changing the next execution plan of the query call plan pointed to by label 301 to the result merging plan pointed to by label 306. "col1>18and" can be removed from the first result filter plans pointed to by labels 304 and 305 respectively. At this point, the updated initial query plan can be as follows: Figure 5 As shown. Among them, Figure 5This is a schematic diagram of the updated initial query plan. Figure 5 The first result filtering plan, pointed to by label 3041, is for... Figure 3 The updated result of the first result filtering plan pointed to by number 304. Figure 5 The first result filtering plan, pointed to by label 3051, is for... Figure 3 The updated result of the first result filtering plan pointed to by number 305.

[0064] In practice, multiple first query plans in the initial query plan may include the same query invocation plan. If this initial query plan is directly used as the data query plan, the same data query operations will be repeatedly executed based on this same query invocation plan during the data query process, thus affecting data query efficiency. Therefore, to effectively improve data query efficiency, when multiple first query plans include the same query invocation plan, they can be merged into a second query plan within the initial query plan. This second query plan includes the same query invocation plan and the merged result of the first result filtering plan for each of the multiple first query plans. The updated initial query plan can then be used as the data query plan.

[0065] In one example, if the second subconditions in multiple conjunctions include the same third subcondition, multiple first query plans in the initial query plan may include the same query invocation plan. To better optimize the initial query plan and further improve data query efficiency, Figure 4 The initial query plan optimization process shown may also include step S407, which is performed before step S409 above.

[0066] In step S407, if multiple first query plans include the same query call plan, the multiple first query plans are merged into a second query plan in the initial query plan. The second query plan includes the same query call plan and the merged result of the first result filtering plan of each of the multiple first query plans.

[0067] by Figure 5 Taking the initial query plan shown as an example, the execution process of step S407 will be illustrated. Figure 5 As shown in the diagram, the query call plan QueryCall(ReadMethod=read_method2, Param='D'), pointed to by label 302, appears twice. Figure 5In the initial query plan shown, the two first query plans, including the query invocation plan, can be merged into a second query plan. This second query plan includes the query invocation plan and a first result filtering plan (such as...) from the two first query plans. Figure 5 The merged result (pointed to by numbers 3041 and 3051 respectively) is the first result filtering plan, which can be Filter(Condition = col5<8 or col6>3). At this point, the updated initial query plan can be as follows: Figure 6 As shown. Among them, Figure 6 This is a schematic diagram of the updated initial query plan. Figure 6 The label 308 in the text points to this merge result.

[0068] Figure 4 The initial query plan optimization process shown avoids multiple executions of filtering operations based on the same conditions by extracting identical filtering conditions from each first result filtering plan. This ensures that filtering operations based on these identical conditions are executed in the next step of the Union operator, reducing the number of calls to the filtering operator. Furthermore, merging multiple first query plans that include the same query call plan prevents the repeated execution of identical data query operations during the data query process. Therefore, data query efficiency can be effectively improved.

[0069] It should be noted that the method for generating the first query plan described above is generally applicable to cases where the conjunctive expression includes a first sub-condition allowed by a single read method. In rare cases, a conjunctive expression in the disjunctive normal form may include first sub-conditions allowed by multiple read methods and second sub-conditions not allowed by the aforementioned read methods. To address this situation, in one implementation, for a given conjunctive expression, a result intersection plan and a result filtering plan including the second sub-condition can be generated. The result intersection plan, the result filtering plan, and the query invocation plans associated with each of the first sub-conditions in the given conjunctive expression are then combined to form a fourth query plan. The result intersection plan may include an intersection operator. In this fourth query plan, the result intersection plan serves as the next execution plan after the query invocation plans associated with each of the first sub-conditions, used to find the intersection of the query results. The result filtering plan serves as the next execution plan after the result intersection plan in this fourth query plan, used to filter the intersection of the query results. It should be understood that this fourth query plan can be used to generate the initial query plan.

[0070] Figure 7 This is a schematic diagram of the data query device in the embodiments of this specification. This device can be applied to any device, platform, or device cluster with data storage, computing, and processing capabilities; for example, it can be applied to a database server (such as...). Figure 1 The database server 102 shown is a data flow hub platform, a privacy collaboration application platform, or a data matrix service platform, etc.

[0071] like Figure 7 As shown, the data query device 700 in this embodiment may include a receiving unit 701, a conversion unit 702, and a generation unit 703. The receiving unit 701 is configured to receive a data query request for a target data table. The data query request includes filtering conditions for multiple fields of the target data table. The target data table supports several reading methods, each of which uses a subset of fields of the target data table for filtering. The conversion unit 702 is configured to convert the filtering conditions into an equivalent disjunctive normal form in response to the presence of logical AND and logical OR operators in the filtering conditions. The generation unit 703 is configured to generate a data query plan based on the disjunctive normal form and the several reading methods.

[0072] In one embodiment, the above-mentioned device 700 may further include: a query unit 704 configured to perform a data query on a target data table based on a data query plan.

[0073] In one embodiment, the transformation unit 702 may be further configured to: convert the above filtering conditions into a matrix expression; calculate the expression; and generate a disjunctive normal form based on the calculation result.

[0074] In one embodiment, the generation unit 703 may be further configured to: extract each of the first sub-conditions allowed by the above disjunction paradigm; for each of the first sub-conditions, generate a query invocation plan based on the first sub-condition and the reading method it satisfies; and generate a data query plan based on the generated query invocation plans.

[0075] In one implementation, the disjunctive paradigm is composed of multiple conjunctions, each of which includes a first sub-condition in each of the first sub-conditions and a second sub-condition that is not allowed by the aforementioned reading methods. The generation unit 703 may be further configured to: for each of the multiple conjunctions, generate a first result filtering plan including the second sub-condition in the conjunction, and combine the first result filtering plan and the query invocation plan associated with the first sub-condition in the conjunction to form a first query plan, wherein the first result filtering plan serves as the next execution plan of the query invocation plan in the first query plan; generate a result merging plan, and combine the result merging plan and each of the first query plans to form an initial query plan, wherein the result merging plan serves as the next execution plan of each of the first query plans in the initial query plan; and generate a data query plan based on the initial query plan.

[0076] In one implementation, any query invocation plan includes a query invocation operator, the method name of the read method used by the query invocation operator, and the value in the first sub-condition associated with the query invocation plan; any first result filtering plan further includes a filtering operator; the result merging plan includes a union operator.

[0077] In one implementation, the generation unit 703 may be further configured to use the initial query plan as the data query plan.

[0078] In one implementation, the second sub-condition in the plurality of conjunctions includes the same third sub-condition; the generation unit 703 may be further configured to: generate a second result filtering plan including the third sub-condition; add the second result filtering plan as the next execution plan of the result merging plan to the initial query plan; update each of the first result filtering plans in the initial query plan so that the updated first result filtering plan no longer contains the third sub-condition; and use the updated initial query plan as the data query plan.

[0079] In one implementation, the plurality of conjunctions include a first conjunction, a second conjunction, and a third conjunction, wherein the third subcondition is the entire content of the second subcondition in the first conjunction and a portion of the second subcondition in each of the second and third conjunctions; the generation unit 703 may be further configured to: remove a first result filtering plan from the first query plan associated with the first conjunction; and remove the third subcondition and a logical AND operator used to connect the third subcondition from the first result filtering plan of the first query plan associated with each of the second and third conjunctions.

[0080] In one implementation, the multiple first query plans in each of the above-mentioned first query plans include the same query invocation plan; the generation unit 703 may be further configured to: merge the multiple first query plans into a second query plan in the initial query plan, the second query plan including the same query invocation plan and the merged result of the first result filtering plan of each of the multiple first query plans; and use the updated initial query plan as the data query plan.

[0081] exist Figure 7 For further explanation of each unit in the corresponding device embodiment, please refer to the relevant descriptions in the previous method embodiments, which will not be repeated here.

[0082] This specification also provides a computer-readable storage medium storing a computer program thereon, wherein when the computer program is executed in a computer, it causes the computer to perform the data query method described in the above method embodiments.

[0083] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the data query method described in the above method embodiments.

[0084] This specification also provides a computer program in its embodiments, wherein when the computer program is executed in a computer, it causes the computer to perform the data query method described in the above method embodiments.

[0085] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0086] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0087] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0088] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes the elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0089] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0090] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0091] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0094] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0095] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0096] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0098] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0099] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A data query method, comprising: Receive a data query request for a target data table, the data query request including filtering conditions for multiple fields of the target data table, the target data table supports several reading methods, and the several reading methods all use some fields of the target data table for filtering and querying; In response to the presence of logical AND and logical OR operators in the filtering conditions, the filtering conditions are converted into equivalent disjunctive normal form. Extract each of the first sub-conditions allowed by the several reading methods from the disjunctive normal form; For each of the first sub-conditions, a query invocation plan is generated based on the first sub-condition and the reading method it satisfies; Based on the generated query call plans, a data query plan is generated.

2. The data query method according to claim 1 further includes: Based on the data query plan, perform a data query on the target data table.

3. The data query method according to claim 1, wherein, The step of converting the filtering conditions into an equivalent disjunctive normal form includes: The filtering conditions are converted into matrix expressions; The expression is calculated, and the disjunctive normal form is generated based on the calculation result.

4. The data query method according to claim 1, wherein, The disjunctive normal form is composed of multiple conjunctions, each of which includes the first sub-condition in each first sub-condition and the second sub-condition that is not allowed by the several reading methods. The process of generating the data query plan based on the generated query invocation plans includes: For each of the plurality of conjunctions, a first result filtering plan including the second sub-condition in the conjunction is generated, and the first result filtering plan and the query invocation plan associated with the first sub-condition in the conjunction are combined to form a first query plan, wherein the first result filtering plan serves as the next execution plan of the query invocation plan in the first query plan; Generate a result merging plan, and combine the result merging plan and each first query plan to form an initial query plan, wherein the result merging plan serves as the next execution plan for each first query plan in the initial query plan; Based on the initial query plan, the data query plan is generated.

5. The data query method according to claim 4, wherein, Any query invocation plan includes a query invocation operator, the method name of the read method used by the query invocation operator, and the value in the first sub-condition associated with the query invocation plan; any first result filtering plan also includes a filtering operator; the result merging plan includes a union operator.

6. The data query method according to claim 4, wherein, The step of generating the data query plan based on the initial query plan includes: The initial query plan is used as the data query plan.

7. The data query method according to claim 4, wherein, The second subcondition in the plurality of conjunctions includes the same third subcondition; The step of generating the data query plan based on the initial query plan includes: Generate a second result filtering plan that includes the third sub-condition; The second result filtering plan is added to the initial query plan as the next execution plan of the result merging plan; In the initial query plan, each first result filtering plan is updated so that the updated first result filtering plan no longer contains the third sub-condition; The updated initial query plan will be used as the data query plan.

8. The data query method according to claim 7, wherein, The plurality of conjunctions include a first conjunction, a second conjunction, and a third conjunction, wherein the third subcondition is the entire content of the second subcondition in the first conjunction and a portion of the second subcondition of each of the second and third conjunctions; The update process for each first result filtering plan in the initial query plan includes: Remove the first result filtering plan from the first query plan of the first conjunctive association; Remove the third subcondition and a logical AND operator used to connect the third subcondition from the first result filtering plan of the first query plan associated with the second and third conjunctions, respectively.

9. The data query method according to claim 4, wherein, The multiple first query plans in each of the first query plans include the same query invocation plan; The step of generating the data query plan based on the initial query plan includes: In the initial query plan, the multiple first query plans are merged into a second query plan, which includes the same query invocation plan and the merged result of the first result filtering plan of each of the multiple first query plans; The updated initial query plan will be used as the data query plan.

10. A data query device, comprising: The receiving unit is configured to receive a data query request for a target data table, the data query request including filtering conditions for multiple fields of the target data table, the target data table supporting several reading methods, and the several reading methods using some fields of the target data table for filtering and querying. The transformation unit is configured to transform the filtering condition into an equivalent disjunctive normal form in response to the presence of logical AND and logical OR operators in the filtering condition. The generation unit is configured to extract each of the first sub-conditions allowed by the plurality of read methods from the disjunction normal form; and for each of the first sub-conditions, generate a query invocation plan based on the first sub-condition and the read method it satisfies. Based on the generated query call plans, a data query plan is generated.

11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-9.

12. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-9.

Images