Data processing method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a data processing method, device, equipment and storage medium. In the data processing method, an associated column is determined from the data table according to the index of the data table, and statistical information of the associated column is calculated based on the data corresponding to the associated column in the data table. Based on this, in the process of calculating the statistical information, the association relationship between the data of different data columns can be fully considered, which effectively improves the closeness between the statistical information of the data table and the actual distribution of the data in the data table.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0002] In the field of database query technology, a query optimizer can create an execution plan corresponding to a database query statement. When creating an execution plan, the query optimizer can calculate the corresponding execution cost and adjust the execution plan according to the execution cost to improve query performance.

[0003] In the prior art, there is a method of obtaining statistical information of data stored in a data table and estimating the execution cost based on the statistical information. However, there is a large deviation between the statistical information obtained by the prior art and the actual distribution of data in the data table, which is not conducive to optimizing the execution plan. Therefore, a solution is urgently needed. Summary of the invention

[0004] Multiple aspects of the present application provide a data processing method, apparatus, device and storage medium to improve the closeness between the statistical information of a data table and the actual distribution of data in the data table.

[0005] An embodiment of the present application provides a data processing method, including: obtaining at least one group of associated columns contained in the data table according to at least one index of the data table; and obtaining statistical information corresponding to the at least one group of associated columns according to data corresponding to the at least one group of associated columns in the data table.

[0006] An embodiment of the present application provides a data processing method, including: obtaining a first number of query conditions included in a first execution plan; determining, among the association columns included in a data table, an association column that matches a query condition group in the first number of query conditions as a target association column; wherein each of the query condition groups includes multiple query conditions; and calculating an execution cost corresponding to the first execution plan based on statistical information corresponding to the target association column.

[0007] An embodiment of the present application also provides a data processing device, including: an associated column determination module, used to obtain at least one group of associated columns contained in the data table according to at least one index of the data table; a statistical information calculation module, used to obtain statistical information corresponding to the at least one group of associated columns according to data in the data table corresponding to the at least one group of associated columns.

[0008] An embodiment of the present application also provides a data processing device, including: a query condition acquisition module, used to obtain a first number of query conditions included in a first execution plan; an associated column acquisition module, used to determine, among the associated columns included in a data table, an associated column that matches a query condition group in the first number of query conditions as a target associated column; wherein each of the query condition groups includes multiple query conditions; and a calculation module, used to calculate the execution cost corresponding to the first execution plan based on statistical information corresponding to the target associated column.

[0009] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to execute the steps in the data processing method provided in the embodiment of the present application.

[0010] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed, the steps in the data processing method provided in the embodiment of the application can be implemented.

[0011] In the data processing method provided in the embodiment of the present application, the associated column is determined from the data table according to the index of the data table, and the statistical information of the associated column is calculated based on the data corresponding to the associated column in the data table. Based on this, in the process of calculating the statistical information, the association relationship between the data in different data columns can be fully considered, which effectively improves the closeness between the statistical information of the data table and the actual distribution of the data in the data table. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0013] Figure 1 A flowchart of a data processing method provided for an exemplary embodiment of the present application;

[0014] Figure 2a A flowchart of a data processing method provided for another exemplary embodiment of the present application;

[0015] Figure 2b A schematic diagram of obtaining statistical information of associated columns provided by an exemplary embodiment of the present application;

[0016] Figure 3 A flowchart of a data processing method provided for another exemplary embodiment of the present application;

[0017] Figure 4 A flowchart of a data processing method provided for another exemplary embodiment of the present application;

[0018] Figure 5a A flowchart of a data processing method provided for another exemplary embodiment of the present application;

[0019] Figure 5b A schematic diagram of statistical information processing logic for multiplexing associated columns and general columns provided by an exemplary embodiment of the present application;

[0020] Figure 6 A schematic diagram of the structure of a data processing device provided for an exemplary embodiment of the present application;

[0021] Figure 7 A schematic diagram of the structure of a data processing device provided for another exemplary embodiment of the present application;

[0022] Figure 8 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present application;

[0023] Fig. 9 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0025] In the prior art, the query optimizer can rely on the statistical information provided by the database to calculate the execution cost corresponding to the execution plan. However, in the prior art, when the database provides statistical information to the optimizer, it often ignores the correlation between different columns in the data table, which leads to a large deviation between the calculation result of the execution cost and the actual result, which is not conducive to screening out a better execution plan.

[0026] In response to this technical problem, an embodiment of the present application provides a data processing method, in which an associated column is determined from a data table according to an index of the data table, and statistical information of the associated column is calculated based on the data corresponding to the associated column in the data table. Based on this, in the process of calculating statistical information, the association relationship between data in different data columns can be fully considered, which effectively improves the closeness between the statistical information of the data table and the actual distribution of data in the data table, which is conducive to effectively improving the accuracy of the estimated execution cost. The following will be specifically described in conjunction with the accompanying drawings.

[0027] Figure 1is a flow chart of a data processing method provided by an exemplary embodiment of the present application, such as Figure 1 As shown, the method includes:

[0028] Step 101: Obtain at least one group of associated columns contained in a data table according to at least one index of the data table.

[0029] Step 102: Obtain statistical information corresponding to the at least one group of associated columns according to the data corresponding to the at least one group of associated columns in the data table.

[0030] In this embodiment, the index of the data table may be created by the user and may include a primary key index, a secondary index, and a unique index (also called a unique constraint). This embodiment includes but is not limited to these.

[0031] Among them, according to one index, a group of associated columns can be selected in the data table; when the data table contains multiple indexes, multiple groups of associated columns can be selected according to the multiple indexes.

[0032] The associated columns refer to multiple data columns in a data table that have an associated relationship. For example, each group of associated columns may include two, three, or more than three related data columns, which is not limited in this embodiment. In this embodiment, the associated relationship of multiple data columns can be identified from the data table according to the index.

[0033] A column in a data table is also called a field in a data table. The existence of an association relationship between multiple data columns can refer to a specific association pattern between the field values ​​corresponding to the multiple different data columns. In some scenarios, the specific pattern can be determined by the relationship between the attribute values ​​represented by the field values ​​in the objective world.

[0034] For example, in an online shopping application scenario, there is an order data table in a relational database. In the order data table, the fields corresponding to the two different columns are the shipping city and the receiving city, and the corresponding field values ​​are: Shanghai and Hangzhou, Shanghai and Urumqi, Shanghai and Lhasa. Since Shanghai and Hangzhou are geographically close in the objective world, users in Hangzhou buy more goods shipped from Shanghai, while users in Urumqi and Lhasa buy fewer goods shipped from Shanghai. Therefore, it can be considered that the field with the field value of Shanghai and the field with the field value of Hangzhou have a high correlation.

[0035] For another example, in a supermarket management application scenario, there is a commodity data table in a relational database. In the commodity data table, the fields corresponding to the two different columns are commodity name and commodity origin, and the corresponding fields are commodity P1 and city C2, commodity P1 and city C2. Assume that commodity P1 is a tropical fruit, city C1 is a tropical city and the main production area of ​​commodity P1, and city C2 is a temperate city and is not the main production area of ​​commodity P1. Then, it can be considered that the frequency of occurrence of the field combination (P1, C1) in the data table is relatively high, so it can be considered that the field with the field value of P1 and the field with the field value of P2 have a high correlation.

[0036] After obtaining at least one group of associated columns from the data table according to the index, the data corresponding to each group of associated columns in the data table may be synthesized to obtain statistical information corresponding to each group of associated columns.

[0037] The statistical information includes statistical information on the distribution of values ​​in one or more columns in a data table, which is used to assist the query optimizer in estimating the cardinality or number of rows in the query result so as to create a high-quality query plan.

[0038] In an embodiment, the statistical information includes but is not limited to one or more of basic statistics, CM-Sketch (Count-Min-Sketch) and histogram. Among them, the basic statistics include but are not limited to the total number, cardinality (NDV), average, maximum value, minimum value, etc. Among them, CM-Sketch is used to estimate the number of times a given value appears. Histograms, including equal-width histograms and equal-height histograms, are used to estimate the frequency of statistics in a certain range.

[0039] In this embodiment, according to the index of the data table, multiple data columns are determined from the data table as associated columns included in the data table, and statistical information of the associated columns is calculated based on the data corresponding to each of the multiple data columns in the data table. Based on this, the association relationship between the data of different data columns can be fully considered in the process of calculating the statistical information, which effectively improves the closeness between the statistical information of the data table and the actual distribution of the data in the data table.

[0040] Figure 2a is a flow chart of a data processing method provided by an exemplary embodiment of the present application, such as Figure 2a As shown, the method includes:

[0041] Step 201: Obtain at least one group of associated columns contained in the data table according to at least one index of the data table.

[0042] Step 202: Obtain statistical information corresponding to the at least one group of associated columns according to the data corresponding to the at least one group of associated columns in the data table.

[0043] Step 203: Obtain a first number of query conditions included in the first execution plan.

[0044] Step 204: Determine, in at least one group of associated columns included in the data table, an associated column that matches a query condition group in the first number of query conditions as a target associated column; wherein each query condition group includes multiple query conditions.

[0045] Step 205: Obtain statistical information of the target associated column from the statistical information corresponding to the at least one group of associated columns.

[0046] Step 206: Calculate the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column.

[0047] In step 201 and step 202, optionally, for any index in the at least one index, a data column may be selected from the data table according to the index. If multiple data columns are selected according to the index, the multiple data columns are used as a group of associated columns included in the data table. A data table may include one or more groups of associated columns, depending on actual conditions.

[0048] Optionally, the index of the data table may include a primary key index, a secondary index, and a unique index (also called a unique constraint). The statistical information corresponding to the associated columns obtained according to different indexes is different. For example, when the index is a "unique index", it is limited by the unique constraint and it can be determined that the data corresponding to the combination of multiple field values ​​only appears once or does not appear. Therefore, it is not necessary to count the CM-Sketch corresponding to the associated columns.

[0049] The following will be combined Figure 2b An example is given. Figure 2b As shown, for a data table, the index created by the user for the data table can be obtained. If the index selects multiple columns from the data table, it can continue to determine whether the index is a unique index. If it is a unique index, the following statistical information can be added based on the data corresponding to the multiple columns: basic statistics corresponding to the multiple columns and histograms corresponding to the multiple columns. If it is not a unique index, the following statistical information can be added based on the data corresponding to the multiple columns: basic statistics corresponding to the multiple columns, CM-Sketch corresponding to the multiple columns and histograms corresponding to the multiple columns, which will not be repeated here.

[0050] Optionally, in this embodiment, according to the data corresponding to the at least one group of associated columns in the data table, before obtaining the statistical information corresponding to the at least one group of associated columns, the data corresponding to the associated columns may be serialized to facilitate the storage of the data corresponding to the associated columns. Optionally, for any group of associated columns in the at least one group of associated columns, the data corresponding to each of the multiple data columns included in the associated column may be obtained; according to the data corresponding to each of the multiple data columns included in the associated column, a tuple corresponding to the associated column is generated, and the tuple is serialized; wherein the tuple includes multiple elements, and the multiple elements respectively correspond to the data of the multiple data columns in the associated column.

[0051] Among them, the optional implementation methods of steps 203 to 205 can be referred to the records of subsequent embodiments and will not be repeated here.

[0052] In this embodiment, according to the index of the data table, multiple data columns are determined from the data table as associated columns included in the data table, and statistical information of the associated columns is calculated based on the data corresponding to each of the multiple data columns in the data table. Based on this, the association relationship between the data of different data columns can be fully considered in the process of calculating the statistical information, which effectively improves the closeness between the statistical information of the data table and the actual distribution of the data in the data table.

[0053] Figure 3 is a flow chart of a data processing method provided by an exemplary embodiment of the present application, such as Figure 3 As shown, the method includes:

[0054] Step 301: Obtain a first number of query conditions included in a first execution plan.

[0055] Step 302: Determine, among the association columns included in the data table, an association column that matches a query condition group in the first number of query conditions as a target association column; wherein each query condition group includes a plurality of query conditions.

[0056] Step 303: Calculate the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column.

[0057] Among them, the execution plan is a description of the execution process or access path of the database query statement in the database. After receiving the database query statement, the query statement can be parsed to obtain a syntax tree, and then the relational expression is converted according to the syntax tree to generate multiple execution plans. In order to improve query efficiency, the query optimizer can predict the execution cost of each execution plan and select a better execution plan from multiple execution plans based on the execution cost. In this embodiment, the first execution plan can be any execution plan whose execution cost is to be calculated, and this embodiment does not limit it.

[0058] The first execution plan includes a first number of query conditions, where the first number is greater than or equal to 2. The number of query condition groups included in the first number of query conditions may be one or more, each query condition group may include multiple query conditions, and each query condition group has a corresponding associated column.

[0059] When determining the associated columns corresponding to the first number of query conditions, one or more query condition groups included in the first number of query conditions can be matched with the associated columns in the data table respectively. For any query condition group, if there is an associated column that matches the query condition group, it can be considered that the multiple different query conditions in the query condition group are non-independent based on the association relationship characterized by the associated column. Compared with the implementation method of assuming that different query conditions are independent of each other for cost estimation, this method of finding query conditions with non-independence from multiple query plans is conducive to obtaining more accurate statistical information, thereby reducing the error of cost estimation.

[0060] In some cases, the first number of query conditions may be divided into a query condition group, and the query condition group includes the first number of query conditions. In this case, if there is an associated column matching the query condition group, it can be considered that all query conditions included in the first execution plan are non-independent.

[0061] In other cases, the first number of query conditions can be divided into multiple query condition groups. In this case, if there are associated columns that match multiple query condition groups respectively, it can be considered that the multiple query conditions in each query condition group are non-independent, and no further details are given.

[0062] Among them, the target association column includes one or more groups of association columns. The number of association columns included in the target association column may be the same as the number of query condition groups included in the first number of query conditions, or may be different, and this embodiment does not limit this. After determining the target association column, the statistical information of the target association column may be calculated according to the data corresponding to the target association column, and the execution cost of the first execution plan may be calculated based on the statistical information, which will not be described in detail.

[0063] In this embodiment, the associated columns are identified in advance from the data table. When calculating the execution cost of the first execution plan, the query conditions included in the first execution plan can be matched with the pre-identified associated columns according to the groups, and the statistical information used to calculate the execution plan can be determined based on the matched associated columns. Furthermore, when calculating the execution cost, the correlation between different columns in the data table can be fully considered, so that the statistical information used to calculate the execution cost is closer to the actual data distribution, which is conducive to effectively improving the accuracy of the estimated execution cost.

[0064] The above embodiment records that, among the association columns included in the data table, an association column matching the query condition group in the first number of query conditions is determined as an implementation method of the target association column. Figure 4 , for further explanation.

[0065] Figure 4 is a flow chart of a data processing method provided by an exemplary embodiment of the present application, such as Figure 4 As shown, the method includes:

[0066] Step 401: Obtain a first number of query conditions included in a first execution plan.

[0067] Step 402: Select a second number of query conditions from the query conditions that are not matched to the associated column and are included in the first number of query conditions to generate an estimated query condition group, where the second number is greater than or equal to 2.

[0068] Step 403 , searching among the associated columns included in the data table whether there is an associated column matching the estimated query condition group; if so, executing step 404 ; if not, executing step 406 .

[0069] Step 404 , obtain the associated columns that match the estimated query condition group, add them to the target associated columns, and execute step 405 .

[0070] Step 405 , determine whether there are multiple query conditions remaining in the first number of query conditions that do not match the associated column; if yes, set the second number to the number of remaining query conditions that do not match the associated column, and execute step 402 ; if no, execute step 407 .

[0071] Step 406: reduce the second number according to the set decreasing step size, and repeat step 402.

[0072] Step 407: Obtain statistical information of the target associated column, and calculate the execution cost corresponding to the first execution plan according to the statistical information of the target associated column.

[0073] In step 401, the first number of query conditions may include all query conditions in the first execution plan, or may include part of the query conditions in the first execution plan, which is not limited in this embodiment.

[0074] In step 402, the estimated query condition group is obtained by assuming the non-independence between multiple query conditions and pre-combining multiple query conditions under the assumption that the second number is greater than or equal to 2, that is, each estimated query condition group includes at least two query conditions.

[0075] The query conditions that are not matched to the associated columns include: query conditions that have not been added to the estimated query condition group for associated column matching, and / or, in the previous associated column matching process, the estimated query condition group to which the query condition group belongs did not match any query conditions of the associated columns.

[0076] In some embodiments, the initial value of the second number may be set to be the same as the value of the first number. That is, when the estimated query condition group is generated for the first time, it may be assumed that the first number of query conditions are not independent of each other.

[0077] Based on this, in the next step 403, if there are associated columns that match the query condition group in the pre-identified associated columns, then it can be considered that the first number of query conditions are not independent of each other. In the subsequent process of calculating statistical information, the association relationship between the first number of query conditions can be fully combined to obtain more accurate statistical information. In the next step 403, if there are no associated columns that match the query condition group in the pre-identified associated columns, step 406 can be executed to reduce the value of the second number according to the set decrement step size to adjust the estimated query conditions included in the query condition group. Optionally, the set decrement step size can be set according to actual needs, and can be set to 1, 2, 3... and so on. Usually, the decrement step size can be set to 1, which will not be repeated.

[0078] Optionally, when executing step 403 to search among the identified associated columns for associated columns that match the estimated query condition group, the estimated query condition group may be matched with each identified associated column one by one, which will not be described in detail.

[0079] When the second number is reduced, the second number is less than the first number. When selecting the second number of query conditions from the query conditions that are not matched to the associated column included in the first number of query conditions, the selection may be performed randomly or according to a set rule, which will be exemplified in combination with different implementations.

[0080] In some exemplary embodiments, when the second number is less than the first number, multiple groups of query conditions of the second number may be selected from the query conditions of the first number of query conditions that are not matched to the associated columns; then, multiple estimated query condition groups are generated based on the multiple groups of query conditions of the second number; wherein, among the multiple estimated query condition groups, different estimated query condition groups contain different query conditions. For example, the first number of query conditions include 4 query conditions that are not matched to the associated columns: a, b, c, d. When the second number is 3, 3 groups of 3 query conditions may be selected from the 4 query conditions, and the generated estimated query condition groups are (a, b, c), (a, b, d), and (b, c, d), etc.

[0081] In this embodiment, when executing step 403, each estimated query condition group can be matched with the associated columns one by one. For example, (a, b, c) is matched with each identified group of associated columns one by one, (a, b, d) is matched with each identified group of associated columns one by one, and (b, c, d) is matched with each identified group of associated columns one by one. If there is one estimated query condition group that matches the associated column among multiple estimated query condition groups, the associated column matched by the estimated query condition group can be added to the target associated column. If there are two or more estimated query condition groups that match the associated column among multiple estimated query condition groups, the associated column matched by any one of the estimated query condition groups can be added to the target associated column; or, according to the priority ranking relationship of the query conditions contained in these two or more estimated query condition groups, one estimated query condition group can be selected from them to add to the target associated column, which will not be repeated.

[0082] In other exemplary embodiments, when the second number is less than the first number, the priority of the query conditions that are not matched to the associated column and are included in the first number of query conditions can be obtained; then, according to the priority, a second number of query conditions whose priorities meet the set requirements are selected from the query conditions that are not matched to the associated column; then, according to the second number of query conditions whose priorities meet the set requirements, an estimated query condition group is generated. In this embodiment, the second number of query conditions generates an estimated query condition group.

[0083] Optionally, the priority of the query condition can be determined according to the priority of the relational operators of different query conditions in the relational algebra corresponding to the first execution plan. For example, the priority of the relational operator of query condition A is greater than the priority of the relational operator of query condition B, and the priority of the relational operator of query condition C is greater than the priority of the relational operator of query condition A. Then, the priority order of query conditions A, B, and C can be considered as: C, A, B.

[0084] Optionally, the priority of the query condition can be determined according to the order of different query conditions in the relational algebra corresponding to the first execution plan. For example, in the relational algebra, the order of three query conditions from left to right is A, B, C, then the priority order of these three query conditions can be considered to be: A, B, C, which will not be repeated.

[0085] Among them, the priority meeting the set requirements may include: the priority meeting the principle of high to low, or the priority meeting the principle of low to high, which is not limited in this embodiment.

[0086] In step 404, if there is a correlation column matching the estimated query condition group among the identified correlation columns, the correlation column can be added as a target correlation column. The statistical information corresponding to the target correlation column can be used to calculate the execution cost.

[0087] Next, step 405 may be performed, and if there are multiple query conditions that do not match the associated column remaining in the first number of query conditions, the remaining multiple query conditions that do not match the associated column may be matched. Optionally, the second number may be set to the number of remaining query conditions that do not match the associated column, so as to preferentially assume that all remaining query conditions are non-independent.

[0088] If there is no query condition that does not match the associated column or there is one query condition that does not match the associated column, step 407 may be executed.

[0089] In the next step 407, statistical information of the target associated column may be obtained, and based on the statistical information, the execution cost corresponding to the first execution plan may be calculated.

[0090] When the first number of query conditions includes M (M is a positive integer) query condition groups, M groups of associated columns may be matched from the identified associated columns as target associated columns, and statistical information of each group of associated columns may be obtained.

[0091] Optionally, if there are no other query conditions in the first number of query conditions except the query condition group matching the associated column, the execution cost of the first execution plan may be directly calculated according to the statistical information corresponding to each group of associated columns in the target associated column.

[0092] If there are other query conditions in the first number of query conditions except the query condition group that matches the associated column, then non-associated columns that match the other query conditions in the data table can be obtained. Then, statistical information corresponding to the non-associated columns is obtained. Then, the execution cost corresponding to the first execution plan can be calculated according to the statistical information corresponding to each group of associated columns and the statistical information corresponding to the non-associated columns, which will not be described in detail.

[0093] In this embodiment, query conditions are selected from the first number of query conditions in order from most to least to form a query condition group, which is conducive to selecting the query condition group with the smallest number and finding the largest number of query conditions with related relationships. Furthermore, when calculating the execution cost, the correlation between different columns in the data table can be fully considered, so that the statistical information used to calculate the execution cost is closer to the actual data distribution, which is conducive to effectively improving the accuracy of the estimated execution cost.

[0094] In the above and the following embodiments of the present application, the associated columns in the data table may be composed of data columns with associated relationships that are pre-identified from the data table.

[0095] Figure 5a is a flow chart of a data processing method provided by another exemplary embodiment of the present application, such as Figure 5a As shown, the method includes:

[0096] Step 501: Identify at least one data column group from a data table, each data column group including a plurality of data columns having an associated relationship.

[0097] Step 502: Use the at least one data column group as at least one group of associated columns included in the data table, and calculate statistical information corresponding to each of the at least one group of associated columns according to the data corresponding to each of the at least one group of associated columns in the data table.

[0098] Step 503: Obtain a first number of query conditions included in the first execution plan.

[0099] Step 504: Determine, in the at least one group of identified association columns, an association column that matches a query condition group in the first number of query conditions as a target association column; wherein each query condition group includes at least one query condition.

[0100] Step 505: Calculate the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column.

[0101] In step 501, optionally, an optional implementation method of identifying association relationships between different data columns in a data table may include: a method of identifying associated columns based on field analysis of columns in a data table, or a method of identifying associated columns based on analysis of indexes of a data table, or a method of identifying associated columns based on analysis of data columns corresponding to historical execution plans of a data table. Specific descriptions are provided below.

[0102] In some embodiments, the associated columns can be identified based on a method of performing field analysis on the columns in the data table. Different columns in the data table represent different attributes, wherein the fields corresponding to the columns represent the attribute names, and the field values ​​corresponding to the columns represent the attribute values. In this embodiment, the correlation of the attribute values ​​represented by the field values ​​of the columns can be analyzed according to a variety of different data query scenarios, and the data columns whose attribute value correlation is greater than the set correlation threshold are used as associated columns.

[0103] For example, in a data table, one column has a product name field, and the field values ​​are Product A and Product B; another column has a store name field, and the field values ​​are Xiao Ming's Store and Product A Specialty Store. Assuming that through the correlation analysis of products and stores, it is determined that the field value "Product A" and the field value "Product A Specialty Store" have a high correlation, the column with the field value "Product A" and the column with the field value "Product A Specialty Store" can be identified as associated columns.

[0104] For another example, in a data table, one column has the shipping city field, and the field values ​​are Shanghai and Guangzhou; another column has the receiving city field, and the field values ​​are Hangzhou and Shenzhen. Assuming that through the correlation analysis of geographic locations, it is determined that the field values ​​"Shanghai" and "Hangzhou" have a high correlation, and the field values ​​"Guangzhou" and "Shenzhen" have a high correlation, then the column with the field value "Shanghai" and the column with the field value "Hangzhou" can be identified as associated columns, and the column with the field value "Guangzhou" and the column with the field value "Shenzhen" can be identified as associated columns.

[0105] In other embodiments, the associated columns of a data table may be identified based on the data columns corresponding to the historical execution plans of the data table. In this embodiment, for the historical execution plan used in the historical query process, the data columns scanned when executing the historical execution plan may be recorded as historical associated columns, and the query conditions contained in the historical execution plan may be recorded as historical query condition groups. Next, a correspondence between historical associated columns and historical query condition groups is established. For example, in a historical query process, a column with a field value of a1 and a column with a field value of a2 are scanned respectively, then the column with a field value of a1 and the column with a field value of a2 may be considered as historical associated columns. When the estimated query condition group contained in the first execution plan is obtained, the first historical query condition group that matches the estimated query condition group may be searched, and based on the correspondence between the historical associated columns and the historical query condition groups, the historical associated columns corresponding to the first historical query condition group may be used as associated columns that match the query condition group.

[0106] In some other embodiments, the associated columns of the data table may be identified based on the analysis of the index of the data table. In this embodiment, a data column may be selected from the data table according to any index of the data table; wherein the index of the data table may be created by the user, and may include a primary key index, a secondary index, a unique index (also called a unique constraint), and this embodiment includes but is not limited to this. If multiple data columns are selected from the data table according to the index, it can be determined that the selected multiple data columns are multiple data columns with an associated relationship, and the multiple data columns with an associated relationship constitute a data column group.

[0107] Of course, in actual implementation, the above implementations can be performed individually or in combination, and this embodiment does not limit it. It should be understood that in addition to the above optional implementations, this application can also adopt other ways of identifying associated columns, which will not be repeated here.

[0108] After obtaining at least one data column group in the data table, step 502 may be executed to use each data column group as a group of associated columns in the data table, and to calculate statistical information corresponding to each group of associated columns according to the data corresponding to each group of associated columns in the data table. The data corresponding to each group of associated columns in the data table is usually user data, which may be logistics data, commodity data, video viewing data, etc., depending on different application scenarios, and this embodiment does not limit this.

[0109] Optionally, in some embodiments, the associated columns and their corresponding data may be recorded and maintained in the form of tuples, which will be described in detail below.

[0110] For any data column group in the at least one data column group, data corresponding to each of the multiple data columns in the data column group that are associated with each other can be obtained from the data table, and a tuple can be generated according to the data corresponding to the multiple data columns in the associated relationship; wherein a group of associated columns corresponds to a tuple, and the tuple contains multiple elements, and the multiple elements respectively correspond to the data of the multiple data columns in the associated relationship in the associated column. Figure 5b As shown, the multiple columns with association relationships selected by the index from the data table are: column 1, column 2...column N (N is a positive integer), and the tuples corresponding to the multiple columns can be expressed as {D(column 1), D(column 2)...D(column N)}, where D(column N) represents the data corresponding to "column N".

[0111] Tuples are a composite, strongly typed data type, and their length is usually fixed; each element in a tuple is of a certain type, and the type of the element is usually a subclass of the data type. Based on the above, tuples can combine data from multiple columns into one data, and can be treated as ordinary data in the database. For example, when calculating statistical information, querying the proportion of December 2008 in the entire data table is equivalent to querying the tuple: (year, month) = (2008, 12), which will not be repeated.

[0112] Based on the tuple method, multi-column data and general column data (usually single-column data) can reuse the statistical information processing logic, which is conducive to reducing the computing cost. The statistical information processing logic includes but is not limited to Figure 5b The calculation, maintenance, update and query processing logic shown are not repeated here.

[0113] Optionally, in this embodiment, in order to enable multiple different types of relational databases to support tuple types, the tuple can be serialized, and the serialized tuple can be saved in the database for subsequent use. When the tuple is serialized, for each element in the tuple, if the data corresponding to the element is fixed-length data, the data corresponding to the element can be directly encoded. In order to prevent the serialized tuple from containing null values, the encoding result of the latter element in the tuple can be directly appended to the encoding result of the previous element, which will not be described in detail.

[0114] If the statistical information corresponding to the element is variable-length data, the statistical information corresponding to the element can be encoded based on the zero-escape encoding method, and a 0 marker value is added at the end of the encoding of each element. In the zero-escape encoding method, 0 is used to mark the end of the encoding of each element. If the encoded data corresponding to the element contains 0, it will be impossible to distinguish the encoded value 0 and the element separator 0 during decoding. Therefore, when encoding the statistical information corresponding to the element based on the zero-escape encoding method, the 0 in the encoded data can be replaced with other numbers, such as 00, so that 0 can be effectively distinguished as an element separator and 00 as the encoded value. For example, the encoding result of the tuple (12,102,303) is (120100203003), where 0 is the element separator and 00 is 0 in the encoded value.

[0115] Based on the data corresponding to each group of associated columns in the data table, the statistical information corresponding to each group of associated columns can be calculated. In this embodiment, for each group of associated columns, the statistical information corresponding to the associated columns is calculated based on the comprehensive data of multiple data columns contained in the associated columns, which is conducive to improving the authenticity and reliability of the statistical information. The following will be exemplified in combination with application scenarios.

[0116] In a typical application scenario, there is an order data table containing multiple order records in a relational database. In the order data table, the fields corresponding to two different columns are the shipping city and the receiving city of the order, and the field values ​​are different city names. When calculating the statistical information of the data table, if the frequency of orders sent from city A to city B in all orders is calculated, then according to the calculation method provided by the prior art, when no associated column is established, the frequency of order records with the shipping city of "city A" P(A) needs to be calculated based on the single column data with the shipping city of "city A" in the data table; and the frequency of order records with the receiving city of "city B" P(B) needs to be calculated based on the single column data with the receiving city of "city B" in the data table; then, the calculated frequency of orders sent from city A to city B P(AB) = P(A)*P(B).

[0117] In some scenarios, the calculated P(AB) deviates from the actual value. For example, based on the rule that users usually choose to ship to the nearest place when shopping, when the distance between city A and city B is relatively close, the number of orders sent from city A to city B is relatively large. In practice, the frequency of orders sent from city A to city B should be higher than P(A)*P(B). When the distance between city A and city B is very far, the number of orders sent from city A to city B is relatively small. In practice, the frequency of orders sent from city A to city B should be lower than P(A)*P(B).

[0118] The associated columns provided based on this embodiment can solve the defects of the above-mentioned prior art. In this embodiment, the column whose field value of the shipping city is city A and the column whose field value of the receiving city is city B in the order data table can be obtained in advance as an associated column, and the tuple {Shipping City: City A, Receiving City: City B} corresponding to the associated column can be obtained. The tuple contains a single column of data with the shipping city as "City A" and a single column of data with the receiving city as "City B". When calculating the frequency of occurrence of orders sent from city A to city B, the frequency P(AB) of order records whose field value of the shipping city is city A and the field value of the receiving city is city B can be obtained from the tuple corresponding to the associated column. The calculated value matches the actual distribution of orders in the data table.

[0119] The optional implementation methods of steps 503 to 505 may refer to the description of the aforementioned embodiment and will not be described in detail here.

[0120] In this embodiment, the associated columns are identified in advance from the data table. When calculating the execution cost of the first execution plan, the query conditions included in the first execution plan can be matched with the pre-identified associated columns according to the groups, and the statistical information used to calculate the execution plan can be determined based on the matched associated columns. Furthermore, when calculating the execution cost, the correlation between different columns in the data table can be fully considered, so that the statistical information used to calculate the execution cost is closer to the actual data distribution, which is conducive to effectively improving the accuracy of the estimated execution cost.

[0121] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 204 can be device A; for another example, the execution subject of steps 201 and 202 can be device A, and the execution subject of step 203 can be device B; and so on.

[0122] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 201, 202, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0123] The above embodiments describe optional implementation methods of the data processing method provided in the embodiments of the present application. In practice, the data processing method can be implemented by a data processing device. The data processing device will be described below in conjunction with the accompanying drawings. For technical details not described in detail in the present embodiments, please refer to the method provided in the embodiments of the present application.

[0124] Figure 6 A structural diagram of a data processing device provided for an exemplary embodiment of the present application is shown in FIG. Figure 6 As shown, the data processing device includes:

[0125] The associated column determination module 601 is used to obtain at least one group of associated columns contained in the data table according to at least one index of the data table. The statistical information calculation module 602 is used to obtain statistical information corresponding to the at least one group of associated columns according to the data corresponding to the at least one group of associated columns in the data table.

[0126] Further optionally, when the associated column determination module 601 obtains at least one group of associated columns contained in the data table according to at least one index of the data table, it is specifically used to: for any index in the at least one index, select a data column from the data table according to the index; if multiple data columns are selected according to the index, then use the multiple data columns as a group of associated columns contained in the data table.

[0127] Further optionally, the statistical information calculation module 602 is also used to: before obtaining the statistical information corresponding to the at least one group of associated columns based on the data corresponding to the at least one group of associated columns in the data table, obtain the data corresponding to each of the multiple data columns contained in the at least one group of associated columns for any one of the at least one group of associated columns; based on the data corresponding to each of the multiple data columns contained in the associated columns, generate a tuple corresponding to the associated column, and serialize the tuple; wherein the tuple contains multiple elements, and the multiple elements respectively correspond to the data of the multiple data columns in the associated column.

[0128] Further optionally, the device also includes an execution cost calculation module 603, which is used to: obtain a first number of query conditions contained in the first execution plan; determine, from at least one group of associated columns contained in the data table, an associated column that matches a query condition group in the first number of query conditions as a target associated column; wherein each of the query condition groups includes multiple query conditions; obtain statistical information of the target associated column from statistical information corresponding to the at least one group of associated columns; and calculate the execution cost corresponding to the first execution plan based on the statistical information corresponding to the target associated column.

[0129] Further optionally, when the execution cost calculation module 603 calculates the execution cost corresponding to the first execution plan based on the statistical information corresponding to the target associated column, it is specifically used to: if there are other query conditions in the first number of query conditions besides the query condition group, then obtain non-associated columns in the data table that self-match the other query conditions; obtain statistical information corresponding to the non-associated columns; and calculate the execution cost corresponding to the first execution plan based on the statistical information corresponding to the target associated columns and the statistical information corresponding to the non-associated columns.

[0130] In this embodiment, according to the index of the data table, multiple data columns are determined from the data table as associated columns included in the data table, and statistical information of the associated columns is calculated based on the data corresponding to each of the multiple data columns in the data table. Based on this, the association relationship between the data of different data columns can be fully considered in the process of calculating the statistical information, which effectively improves the closeness between the statistical information of the data table and the actual distribution of the data in the data table.

[0131] Figure 7 A structural diagram of a data processing device provided for an exemplary embodiment of the present application is shown in FIG. Figure 7 As shown, the data processing device includes: a query condition acquisition module 701, a related column acquisition module 702 and a calculation module 703.

[0132] The query condition acquisition module 701 is used to acquire a first number of query conditions included in the first execution plan;

[0133] The associated column acquisition module 702 is used to determine, among the associated columns included in the data table, associated columns that match the query condition groups in the first number of query conditions as target associated columns; wherein each of the query condition groups includes multiple query conditions;

[0134] The calculation module 703 is used to calculate the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column.

[0135] Further optionally, the associated column acquisition module 702 determines, among the associated columns included in the data table, an associated column that matches the query condition group in the first number of query conditions, and, when used as a target associated column, is specifically used to: select a second number of query conditions from the query conditions that are not matched to the associated columns and included in the first number of query conditions, to generate an estimated query condition group; search, among the associated columns included in the data table, whether there is an associated column that matches the estimated query condition group; if so, obtain the associated column that matches the estimated query condition group and add it to the target associated column; if not, reduce the second number, and repeat the step of selecting a second number of query conditions that are not matched to the associated columns from the first number of query conditions.

[0136] Further optionally, when the associated column acquisition module 702 selects a second number of query conditions from the query conditions that are not matched to the associated columns and included in the first number of query conditions to generate an estimated query condition group, it is specifically used to: if the second number is less than the first number, select multiple groups of the second number of query conditions from the query conditions that are not matched to the associated columns; generate multiple estimated query condition groups based on the multiple groups of the second number of query conditions; wherein, among the multiple estimated query condition groups, different estimated query condition groups contain different query conditions.

[0137] Further optionally, when the associated column acquisition module 702 selects a second number of query conditions from the query conditions that are not matched to the associated column and included in the first number of query conditions to generate an estimated query condition group, it is specifically used to: if the second number is less than the first number, obtain the priorities of the query conditions that are not matched to the associated column; based on the priorities, select a second number of query conditions whose priorities meet the set requirements from the query conditions that are not matched to the associated column; and generate the estimated query condition group based on the second number of query conditions whose priorities meet the set requirements.

[0138] Further optionally, the calculation module 703 includes: a statistical information calculation submodule 7031 and a cost calculation submodule 7032; when the calculation module 703 calculates the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target associated column, if there are other query conditions in the first number of query conditions except the query condition group, then the statistical information calculation submodule 7031 is used to obtain non-associated columns in the data table that self-match the other query conditions; the statistical information calculation submodule 7031 is used to obtain the statistical information corresponding to each group of associated columns in the target associated column and the statistical information corresponding to the non-associated columns, and the cost calculation submodule 7032 is used to calculate the execution cost corresponding to the first execution plan according to the statistical information corresponding to each group of associated columns and the statistical information corresponding to the non-associated columns.

[0139] Further optionally, if Figure 7 As shown, the device further includes: an associated column processing module 704; the associated column processing module 704 is used to: identify at least one data column group from the data table, each data column group includes multiple data columns with an associated relationship; use the at least one data column group as at least one group of associated columns included in the data table; and calculate the statistical information corresponding to each of the at least one group of associated columns according to the data corresponding to each of the at least one group of associated columns in the data table. .

[0140] Further optionally, the preprocessing module 704 identifies at least one data column group from a data table, and when each data column group contains multiple data columns with an associated relationship, it is specifically used to: for any index of the data table, select a data column from the data table according to the index; if multiple data columns are selected from the data table, determine that the selected multiple data columns are a data column group.

[0141] Further optionally, the associated column processing module 704 is also used to: for any data column group in the at least one data column group, obtain the data corresponding to each of the multiple data columns with an associated relationship in the data column group; generate a tuple according to the data corresponding to each of the multiple data columns with an associated relationship; serialize the tuple to save the serialized tuple in a database; wherein the tuple contains multiple elements, and the multiple elements respectively correspond to the data of the multiple data columns with an associated relationship.

[0142] In this embodiment, the associated columns are identified in advance from the data table. When calculating the execution cost of the first execution plan, the query conditions included in the first execution plan can be matched with the pre-identified associated columns according to the groups, and the statistical information used to calculate the execution plan can be determined based on the matched associated columns. Furthermore, when calculating the execution cost, the correlation between different columns in the data table can be fully considered, so that the statistical information used to calculate the execution cost is closer to the actual data distribution, which is conducive to effectively improving the accuracy of the estimated execution cost.

[0143] Figure 8 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, such as Figure 8 As shown, the electronic device includes: a memory 801 and a processor 802.

[0144] The memory 801 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0145] The memory 801 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0146] The processor 802 is coupled to the memory 801 and is used to execute a computer program in the memory 801, so as to: obtain at least one group of associated columns contained in the data table according to at least one index of the data table; and obtain statistical information corresponding to the at least one group of associated columns according to data corresponding to the at least one group of associated columns in the data table.

[0147] Further optionally, when the processor 802 obtains at least one group of associated columns contained in the data table based on at least one index of the data table, it is specifically used to: for any index in the at least one index, select a data column from the data table according to the index; if multiple data columns are selected according to the index, then use the multiple data columns as a group of associated columns contained in the data table.

[0148] Further optionally, before obtaining statistical information corresponding to the at least one group of associated columns based on the data corresponding to the at least one group of associated columns in the data table, the processor 802 is also used to: obtain, for any group of associated columns in the at least one group of associated columns, data corresponding to each of the multiple data columns contained in the associated column; generate a tuple corresponding to the associated column based on the data corresponding to each of the multiple data columns contained in the associated column, and serialize the tuple; wherein the tuple contains multiple elements, and the multiple elements respectively correspond to the data of the multiple data columns in the associated column.

[0149] Further optionally, the processor 802 is also used to: obtain a first number of query conditions contained in the first execution plan; determine, from at least one group of associated columns contained in the data table, an associated column that matches a query condition group in the first number of query conditions as a target associated column; wherein each of the query condition groups contains multiple query conditions; obtain statistical information of the target associated column from statistical information corresponding to the at least one group of associated columns; and calculate the execution cost corresponding to the first execution plan based on the statistical information corresponding to the target associated column.

[0150] Further optionally, when the processor 802 calculates the execution cost corresponding to the first execution plan based on the statistical information corresponding to the target associated column, it is specifically used to: if there are other query conditions besides the query condition group among the first number of query conditions, obtain non-associated columns in the data table that self-match the other query conditions; obtain statistical information corresponding to the non-associated columns; and calculate the execution cost corresponding to the first execution plan based on the statistical information corresponding to the target associated columns and the statistical information corresponding to the non-associated columns.

[0151] Further, if Figure 8 As shown, the electronic device also includes: a communication component 803, a display 804, a power component 805, an audio component 806 and other components. Figure 8 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 8 Components shown.

[0152] In this embodiment, according to the index of the data table, multiple data columns are determined from the data table as associated columns included in the data table, and statistical information of the associated columns is calculated based on the data corresponding to each of the multiple data columns in the data table. Based on this, the association relationship between the data of different data columns can be fully considered in the process of calculating the statistical information, which effectively improves the closeness between the statistical information of the data table and the actual distribution of the data in the data table.

[0153] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be performed by an electronic device in the above method embodiment.

[0154] Fig. 9 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, such as Fig. 9 As shown, the electronic device includes: a memory 901 and a processor 902 .

[0155] The memory 901 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0156] Memory 901 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0157] Processor 902 is coupled to memory 901 and is used to execute a computer program in memory 901, so as to: obtain a first number of query conditions contained in a first execution plan; determine, among the association columns contained in a data table, an association column that matches a query condition group in the first number of query conditions as a target association column; wherein each of the query condition groups contains multiple query conditions; and calculate an execution cost corresponding to the first execution plan based on statistical information corresponding to the target association column.

[0158] Further optionally, the processor 902 determines, in the association columns contained in the data table, an association column that matches the query condition group in the first number of query conditions, and uses it as a target association column, and is specifically used to: select a second number of query conditions from the query conditions that are not matched to the association columns contained in the first number of query conditions, and generate an estimated query condition group; in the association columns contained in the data table, search whether there is an association column that matches the estimated query condition group; if so, obtain the association column that matches the estimated query condition group and add it to the target association column; if not, reduce the second number, and repeat the step of selecting a second number of query conditions that are not matched to the association column from the first number of query conditions.

[0159] Further optionally, when the processor 902 selects a second number of query conditions from the query conditions that are not matched to the associated column and includes the first number of query conditions to generate an estimated query condition group, it is specifically used to: if the second number is less than the first number, select multiple groups of the second number of query conditions from the query conditions that are not matched to the associated column; generate multiple estimated query condition groups based on the multiple groups of the second number of query conditions; wherein, among the multiple estimated query condition groups, different estimated query condition groups contain different query conditions.

[0160] Further optionally, when the processor 902 selects a second number of query conditions from the query conditions that are not matched to the associated column included in the first number of query conditions to generate an estimated query condition group, it is specifically used to: if the second number is less than the first number, obtain the priorities of the query conditions that are not matched to the associated column; based on the priorities, select a second number of query conditions whose priorities meet the set requirements from the query conditions that are not matched to the associated column; and generate the estimated query condition group based on the second number of query conditions whose priorities meet the set requirements.

[0161] Further optionally, when the processor 902 calculates the execution cost corresponding to the first execution plan based on the statistical information corresponding to the target associated columns, it is specifically used to: if there are other query conditions besides the query condition group in the first number of query conditions, then obtain non-associated columns in the data table that self-match the other query conditions; obtain the statistical information corresponding to each group of associated columns in the target associated columns and the statistical information corresponding to the non-associated columns; calculate the execution cost corresponding to the first execution plan based on the statistical information corresponding to each group of associated columns and the statistical information corresponding to the non-associated columns.

[0162] Further optionally, the processor 902 is further configured to identify at least one data column group from the data table, each data column group including a plurality of data columns having an associated relationship; use the at least one data column group as at least one group of associated columns included in the data table; and calculate statistical information corresponding to each of the at least one group of associated columns according to the data corresponding to each of the at least one group of associated columns in the data table.

[0163] Further optionally, when the processor 902 identifies at least one data column group from a data table, and each data column group includes multiple data columns with an associated relationship, it is specifically used to: for any index of the data table, select a data column from the data table according to the index; if multiple data columns are selected from the data table, determine that the selected multiple data columns are a data column group.

[0164] Further optionally, the processor 902 is also used to: for any data column group in the at least one data column group, obtain data corresponding to each of the multiple data columns that are in an associated relationship in the data column group; generate a tuple based on the data corresponding to each of the multiple data columns that are in an associated relationship; serialize the tuple to save the serialized tuple in a database; wherein the tuple contains multiple elements, and the multiple elements respectively correspond to the data of the multiple data columns that are in an associated relationship.

[0165] Further, if Fig. 9 As shown, the electronic device also includes: a communication component 903, a display 904, a power component 905, an audio component 906 and other components. Fig. 9 Only some components are shown schematically, which does not mean that the electronic device only includes Fig. 9 Components shown.

[0166] In this embodiment, the associated columns are identified in advance from the data table. When calculating the execution cost of the first execution plan, the query conditions included in the first execution plan can be matched with the pre-identified associated columns according to the groups, and the statistical information used to calculate the execution plan can be determined based on the matched associated columns. Furthermore, when calculating the execution cost, the correlation between different columns in the data table can be fully considered, so that the statistical information used to calculate the execution cost is closer to the actual data distribution, which is conducive to effectively improving the accuracy of the estimated execution cost.

[0167] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0168] exist Figure 8 as well as Fig. 9 In the corresponding embodiments, the processor can be implemented as a central processing unit (CPU), a graphics processing unit (GPU) or a microcontroller unit (MCU); it can also be implemented as a field programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), a complex programmable logic device (CPLD) and other programmable devices; or it can be implemented as an advanced reduced instruction set (RISC) processor (Advanced RISCMachines, ARM) or a system on chip (System on Chip SOC), etc., but not limited to this.

[0169] exist Figure 8 as well as Fig. 9 In a corresponding embodiment, the communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0170] exist Figure 8 as well as Fig. 9 In a corresponding embodiment, the display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0171] exist Figure 8 as well as Fig. 9 In a corresponding embodiment, the power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0172] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0177] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0178] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0179] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0180] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: According to at least one index of the data table, obtaining at least one group of associated columns contained in the data table; Acquire statistical information corresponding to the at least one group of associated columns according to the data in the data table corresponding to the at least one group of associated columns; The method further includes: obtaining a first number of query conditions included in the first execution plan; determining, from at least one group of associated columns included in the data table, associated columns that match a query condition group in the first number of query conditions as target associated columns; wherein each query condition group includes multiple query conditions; obtaining statistical information of the target associated columns from statistical information corresponding to the at least one group of associated columns; and calculating the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target associated columns; The method further includes: for any group of associated columns, obtaining data corresponding to each of the multiple data columns included in the associated columns; generating tuples corresponding to the associated columns according to the data corresponding to each of the multiple data columns included in the associated columns, and serializing the tuples, so as to record and maintain the associated columns and their corresponding data through the serialized tuples.

2. The method according to claim 1, characterized in that According to at least one index of a data table, obtaining at least one group of associated columns contained in the data table includes: For any index of the at least one index, select a data column from the data table according to the index; If multiple data columns are selected according to the index, the multiple data columns are used as a group of associated columns included in the data table.

3. The method according to claim 1, characterized in that The tuple includes a plurality of elements, and the plurality of elements respectively correspond to data of a plurality of data columns in the associated column.

4. The method according to claim 1, characterized in that: Calculating the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column includes: If, among the first number of query conditions, there are other query conditions besides the query condition group, obtaining non-associated columns in the data table that are self-matching with the other query conditions; Obtaining statistical information corresponding to the non-associated column; The execution cost corresponding to the first execution plan is calculated according to the statistical information corresponding to the target associated column and the statistical information corresponding to the non-associated column.

5. A data processing method, characterized in that: include: Obtain a first number of query conditions included in a first execution plan; Determine, among the association columns included in the data table, an association column that matches a query condition group in the first number of query conditions as a target association column; wherein each query condition group includes a plurality of query conditions; Calculating the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column; The method also includes: for any group of associated columns in the associated columns contained in the data table, obtaining data corresponding to each of the multiple data columns contained in the associated columns; generating tuples corresponding to the associated columns according to the data corresponding to each of the multiple data columns contained in the associated columns, and serializing the tuples to record and maintain the associated columns and their corresponding data through the serialized tuples.

6. The method according to claim 5, characterized in that Determining, from the association columns included in the data table, an association column that matches the query condition group in the first number of query conditions as a target association column includes: Selecting a second number of query conditions from the query conditions that are not matched to the associated columns and are included in the first number of query conditions to generate an estimated query condition group; Searching among the associated columns included in the data table whether there is an associated column matching the estimated query condition group; If so, obtain the associated column matching the estimated query condition group and add it to the target associated column; If not, the second number is reduced, and the step of selecting a second number of query conditions that do not match the associated column from the first number of query conditions is repeated.

7. The method according to claim 6, characterized in that Selecting a second number of query conditions from the query conditions that are not matched to the associated columns and are included in the first number of query conditions to generate an estimated query condition group includes: If the second number is less than the first number, selecting multiple groups of query conditions of the second number from the query conditions that are not matched to the associated column; A plurality of the estimated query condition groups are generated according to the plurality of groups of the second number of query conditions; wherein different estimated query condition groups among the plurality of the estimated query condition groups contain different query conditions.

8. The method according to claim 6, characterized in that Selecting a second number of query conditions from the query conditions that are not matched to the associated columns and are included in the first number of query conditions to generate an estimated query condition group includes: If the second number is less than the first number, obtaining the priority of the query condition that is not matched to the associated column; According to the priority, selecting a second number of query conditions whose priorities meet the set requirements from the query conditions that are not matched to the associated column; The estimated query condition group is generated according to the query conditions of the second number of priorities that meet the set requirements.

9. The method according to claim 5, characterized in that Calculating the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column includes: If, among the first number of query conditions, there are other query conditions besides the query condition group, obtaining non-associated columns in the data table that are self-matching with the other query conditions; Obtaining statistical information corresponding to each group of associated columns in the target associated columns and statistical information corresponding to the non-associated columns; The execution cost corresponding to the first execution plan is calculated according to the statistical information corresponding to each group of associated columns and the statistical information corresponding to the non-associated columns.

10. The method according to any one of claims 5 to 9, characterized in that: Also includes: Identify at least one data column group from the data table, each data column group comprising a plurality of data columns having an associated relationship; Using the at least one data column group as at least one group of associated columns included in the data table; According to the data respectively corresponding to the at least one group of associated columns in the data table, statistical information respectively corresponding to the at least one group of associated columns is calculated.

11. The method according to claim 10, characterized in that At least one data column group is identified from the data table, each data column group comprising a plurality of data columns having an associated relationship, including: For any index of the data table, select a data column from the data table according to the index; If a plurality of data columns are selected from the data table, the plurality of data columns selected are determined to be a data column group.

12. A data processing device, characterized in that: include: An associated column determination module, configured to obtain at least one group of associated columns contained in the data table according to at least one index of the data table; A statistical information calculation module, used for obtaining statistical information corresponding to the at least one group of associated columns according to the data corresponding to the at least one group of associated columns in the data table; The device also includes a cost calculation module, which is used to obtain a first number of query conditions included in the first execution plan; Determine, from the at least one group of associated columns included in the data table, an associated column that matches a query condition group in the first number of query conditions as a target associated column; wherein each query condition group includes multiple query conditions; obtain statistical information of the target associated column from the statistical information corresponding to the at least one group of associated columns; and calculate the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target associated column; The statistical information calculation module is also used to: for any group of associated columns, obtain the data corresponding to each of the multiple data columns contained in the associated columns; generate tuples corresponding to the associated columns based on the data corresponding to each of the multiple data columns contained in the associated columns, and serialize the tuples to record and maintain the associated columns and their corresponding data through the serialized tuples.

13. A data processing device, characterized in that: include: A query condition acquisition module, used to acquire a first number of query conditions included in a first execution plan; An associated column acquisition module, used to determine, among the associated columns included in the data table, associated columns that match the query condition groups in the first number of query conditions as target associated columns; wherein each of the query condition groups includes multiple query conditions; A calculation module, used for calculating the execution cost corresponding to the first execution plan according to the statistical information corresponding to the target association column; The device also includes an associated column processing module, which is used to: for any group of associated columns in the associated columns contained in the data table, obtain the data corresponding to each of the multiple data columns contained in the associated columns; generate a tuple corresponding to the associated column according to the data corresponding to each of the multiple data columns contained in the associated columns, and serialize the tuple to record and maintain the associated column and its corresponding data through the serialized tuple.

14. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to perform the steps in the method according to any one of claims 1-11.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the steps in the method according to any one of claims 1 to 11 can be implemented.

Images