Method of obtaininig distribution key and related device

Abstract

Embodiments of the present application provide a method of obtaining a distribution key and related devices for obtaining a distribution key. The method is applied to cloud management platforms. This method includes: receiving a request from a tenant, where the request is used to obtain a set of target distribution keys; obtaining a set of target tables and a set of candidate distribution keys based on the request and a set of historical query information; obtaining a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule; and obtaining the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys. The proposed technique provides a suitable distribution key for tenant, improves the efficiency of obtaining a distribution key, and improves the efficiency of query.

Description

METHOD OF OBTAINING DISTRIBUTION KEY AND RELATEDDEVICETECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of cloud computing, and more particularly, to a method of obtaining a distribution key, and related devices for obtaining a distribution key.BACKGROUND

[0002] Distributed storage technology in the field of cloud computing stores data of tenant in multiple storage nodes to protect data security and support parallel queries, thereby reducing the time for tenants to query data. When the tenant's data is represented in the form of a table, the table includes multiple columns, and each column in the table corresponds to a data attribute. A cloud vendor stores the table based on the distribution key selected by the tenant. For example, if the table includes multiple data attributes such as "name", "gender", "age", etc., the tenant uses "name" as a distribution key and performs hash operations on each data corresponding to the data attribute named "name", thereby storing data with the same hash value in the same storage node. When the tenant has a large number of tables and multiple tables are associated, the tenant uses a join query statement to query data. For example, if the tenant queries all the data of a person named "Zhang" in table 1 and table 2 through a join query statement, the join key in the join query statement is the "name". If table 1 and table 2 are stored in different storage nodes, when data is retrieved, it is necessary to access the storage node 2 where table 2 is stored, in order to obtain all the data of the person named "Zhang" in table 2. Therefore, in order to reduce the time of query, "name" is used as a distribution key, so that the data with the same hash value in the same column of the tenant's table is stored in the same storage node, thereby saving time of accessing other storage node and transmitting data. However, in situations wherethe tenant has a large number of tables and each table contains a large number of data attributes, it is difficult for the tenant to manually determine the appropriate distribution key, resulting in a longer time of query for the tenant. Moreover, manually determining the distribution key requires a significant amount of time and labor costs, and determining the distribution key through mathematical models is difficult to adapt to various practical application scenarios.

[0003] Therefore, how to provide a suitable distribution key for a tenant, improve the efficiency of obtaining a distribution key, and improve the efficiency of query is a challenge.SUMMARY

[0004] Embodiments of the present application provide a method of obtaining a distribution key and related devices for obtaining a distribution key. The technical solution provides a suitable distribution key for a tenant, improves the efficiency of obtaining a distribution key, and improves the efficiency of query.

[0005] According to a first aspect, an embodiment of this application provides a method of obtaining a distribution key. This method is applied to a cloud management platform, the cloud management platform manages an infrastructure used to provide cloud services, the infrastructure includes at least one data center, each data center in the at least one data center includes storage nodes, and this method includes: receiving a request from a tenant, where the request is used to obtain a set of target distribution keys, and each target distribution key in the set of target distribution keys is used to store a set of tables of the tenant in the storage nodes; obtaining a set of target tables and a set of candidate distribution keys based on the request and a set of historical query information, where the set of historical query information includes at least one historical join query statement, each historical join query statement of the at least one historical join query statement is used to query data in at least two target tables in the set of target tables, and a join key in each historical join query statement is a candidate distribution key for each target table corresponding to each historical join query statement; obtaining a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule, where the first optimization rule includes a set of preset conditions corresponding to a set of first attributes of a distribution key, each preset condition in the set of preset conditionscorresponds to a first attribute in the set of first attributes, and each first attribute corresponding to the set of first distribution keys satisfies the preset condition corresponding to the first attribute; and obtaining the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys, where each target distribution key in the set of target distribution keys satisfies a first constraint condition, each target table in the set of target tables corresponds to a target distribution key in the set of target distribution keys, each target distribution key in the set of target distribution keys is a single-column distribution key or a multi-column distribution key, the network cost of each couple of first distribution keys in the set of first distribution keys is determined based on the network cost of at least one historical join query statement corresponding to the couple of first distribution keys, and the couple of first distribution keys is determined based on the at least one historical join query statement.

[0006] According to the method above, the cloud management platform removes the distribution key that does not satisfy the preset condition included in the first optimization rule from the set of candidate distribution keys, thereby reducing the number of potential distribution keys and improving the efficiency of obtaining the set of target distribution keys. Meanwhile, the first optimization rule can be flexibly adjusted, allowing the tenant or cloud vendor to flexibly exclude at least one inappropriate distribution key from the set of candidate distribution keys based on practical application scenarios. Moreover, according to this method, at least one target distribution key that satisfies the first constraint conditions is determined from the set of first distribution keys based on the network cost of each couple of first distribution keys, and one or more target distribution key is determined for each target table, thereby minimizing the network cost of querying data after storing data based on the set of target distribution keys and improving the efficiency of querying data.

[0007] In a possible design, when the target distribution key is a single-column distribution key, the target distribution key is determined based on a first distribution key. When the target distribution key is a multi-column distribution key, the target distribution key is determined based on multiple first distribution keys.

[0008] For example, storing a first table based on a first single-column distribution key includes: hashing the data in a first column of the first table and storing data with the same hashvalue in the same storage node. The first single-column distribution key corresponds to the first column in the first table, and the first table includes at least one column. In other words, each row in the first table corresponds to a hash value, which is determined by hashing the data in the first column of each row.

[0009] For example, storing the first table based on a first multi-column distribution key includes: hashing the data in the first column and second column of the first table, and storing data with the same hash value in the same storage node. The first multi-column distribution key corresponds to the first column and second column in the first table, and the first table includes multiple columns. In other words, each row in the first table corresponds to a hash value, which is determined by hashing the data in the first column and second column of each row.

[0010] In a possible design, the set of first attributes includes: estimated value of data skewness and / or query running time, the estimated value of data skewness corresponding to each candidate distribution key is used to estimate the unevenness of distribution on data when at least one target table corresponding to each candidate distribution key is stored in the storage nodes based on the candidate distribution key, and the query running time corresponding to the set of candidate distribution keys includes each time required to query according to each historical join query statement after storage of the set of target tables is simulated based on the set of candidate distribution keys.

[0011] In a possible design, the query running time corresponding to the set of candidate distribution keys is obtained by redistributing the set of target tables based on the set of candidate distribution keys. Alternatively, the query running time corresponding to the set of candidate distribution keys is obtained by simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the query running time corresponding to the set of candidate distribution keys is obtained by estimating manually by technician (e.g. database management system (DBMS) planner).

[0012] In a possible design, the obtaining a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule includes: obtaining the set of first distribution keys from the set of candidate distribution keys based on the estimated value of data skewness and / or query running time corresponding to the set of candidate distribution keys, where the estimated value of data skewness of each first distribution key is less than orequal to a first preset threshold, and / or, the query running time corresponding to the set of first distribution keys satisfies a first preset condition in the set of preset conditions.

[0013] According to the method above, the cloud management platform removes the distribution key with higher estimated value of data skewness from the set of candidate distribution keys, thereby avoiding the tenant from storing the table based on the distribution key with higher estimated value of data skewness. Moreover, this method simulates the query running time corresponding to the set of candidate distribution keys to avoid obtaining the distribution key with a longer query running time for the tenant, thereby improving the efficiency of querying data after storing data based on the set of target distribution keys.

[0014] In a possible design, the obtaining the set of first distribution keys from the set of candidate distribution keys based on the estimated value of data skewness and / or query running time corresponding to the set of candidate distribution keys includes: obtaining a set of second distribution keys from the set of candidate distribution keys based on the estimated value of data skewness of each candidate distribution key, where the estimated value of data skewness of each second distribution key is less than or equal to a second preset threshold; and obtaining the set of first distribution keys from the set of second distribution keys based on the query running time corresponding to the set of second distribution keys, where the query running time corresponding to the set of first distribution keys satisfies the first preset condition.

[0015] According to the method above, the cloud management platform removes the distribution key with higher estimated value of data skewness from the set of candidate distribution keys. Then, the cloud management platform obtains the distribution key corresponding to a shorter query running time from the remaining candidate distribution keys, thereby reducing the number of first distribution keys, saving the computational cost of determining the target distribution key and improving computational efficiency.

[0016] In a possible design, the obtaining the set of first distribution keys from the set of second distribution keys based on the query running time corresponding to the set of second distribution keys includes: when the query running time corresponding to the set of second distribution keys does not satisfy the first preset condition, obtaining the set of first distribution keys from the set of second distribution keys, where the estimated value of data skewness of each first distribution key is less than or equal to a third preset threshold, and the third presetthreshold is less than the second preset threshold; or, when the query running time corresponding to the set of second distribution keys satisfies the first preset condition, determining the set of second distribution keys as the set of first distribution keys.

[0017] In a possible design, the first preset condition includes at least one of the following: a sum of query running time corresponding to the set of first distribution keys is less than or equal to a fourth preset threshold; or, a sum of query running time corresponding to the set of first distribution keys is less than or equal to a sum of query running time corresponding to a set of original distribution keys; or, a difference between a sum of query running time corresponding to the set of first distribution keys and a sum of query running time corresponding to a set of original distribution keys is less than or equal to a fifth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a sixth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a second query running time corresponding to a set of original distribution keys; or, a difference between a first query running time corresponding to the set of first distribution keys and a second query running time corresponding to a set of original distribution keys is less than or equal to a seventh preset threshold; where each original distribution key in the set of original distribution keys is a distribution key used by the tenant to store the set of tables, the first query running time is determined based on the query running time corresponding to the set of first distribution keys, and the second query running time is determined based on the query running time corresponding to a set of original distribution keys.

[0018] In a possible design, the sum of query running time corresponding to the set of first distribution keys is obtained by querying each historical join query statement after redistributing the set of target tables based on the set of first distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained by querying each historical join query statement after simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained by estimating manually by technician (e.g. DBMS planner).

[0019] According to the method above, when the cloud management platform removes the inappropriate distribution key from the set of second distribution keys through the queryrunning time, the distribution key with higher estimated value of data skewness in the set of second distribution keys is removed, thereby ensuring that the query runtime corresponding to the remaining distribution keys is not too long, or the query runtime corresponding to the remaining distribution keys is shorter than that corresponding to the set of original distribution keys.

[0020] In a possible design, the first constraint condition includes: the set of target distribution keys includes each target distribution key corresponding to each target table in the set of target tables, and each target distribution key is determined based on at least one first distribution key; the at least one first distribution key corresponding to a target distribution key corresponding to a first target table is determined based on at most one couple of first distribution keys corresponding to the first target table and each second target table in the set of second target tables, the first target table belongs to the set of target tables, each second target table and the first target table are associated target tables, and the associated target tables are tables queried through the same historical join query statement in the set of historical query information; and each first distribution key corresponding to the target distribution key is the distribution key in the set of first distribution keys that maximizes a value of>where A is the set of target tables, a and b are different target tables in A, W‘bis a network cost of a ith couple of first distribution keys corresponding to a and b, Xa'bis a first value of the ith couple of first distribution keys corresponding to a and b, Wbis a non-negative number, 0 < Xa‘b< 1 , 1 < i < I* , lab is the number of couple of first distribution keys corresponding to a and b, and Iab is a positive integer.

[0021] In a possible design, the value of the Xa‘bis any of the following: 0, 1, a decimal between 0 and 1.

[0022] According to the method above, the cloud management platform obtains at least one first distribution key corresponding to each target table based on the first constraint condition, and the sum of the network cost of each couple of first distribution keys should be as high as possible. Therefore, when the set of target tables is stored based on the set of target distribution keys, the network cost of corresponding couple of target distribution keys is saved, therebyimproving the efficiency of querying.

[0023] In a possible design, the first value of each first distribution key corresponding to the target distribution key is greater than or equal to an eighth preset threshold, the first value of each first distribution key is a first value of a couple of first distribution keys which includes each first distribution key, and the method further includes: providing the tenant with the set of target distribution keys, or, providing the tenant with the set of target distribution keys and the first value of each couple of first distribution keys corresponding to each target distribution key.

[0024] According to the method above, the cloud management platform provides the tenant with the set of target distribution keys that satisfies the first constraint condition and corresponds to at least one first distribution key with a high first value. And, the cloud management platform also provides the tenant with the first value of each couple of target distribution keys, so that the tenant has more choices for the distribution key.

[0025] In a possible design, the method further includes: obtaining a statistic of each target table from a database of the tenant, where the statistic of each target table includes at least one of the following: a size of the target table, a percentage of an undefined value in the target table, or the number of unique values in the target table; and obtaining the estimated value of data skewness of each candidate distribution key based on the statistic of each target table and the set of candidate distribution keys.

[0026] According to the method above, the cloud management platform obtains the estimated value of data skewness of each distribution key based on the statistic of the corresponding target table, and does not need to store the set of tables based on the set of candidate distribution keys, thereby improving the efficiency of obtaining the set of target distribution keys.

[0027] In a possible design, the method further includes: providing the tenant with at least one of the following: at least one value corresponding to at least one first attribute of each first distribution key, the network cost of each couple of first distribution keys, a first storage statement, a sequence of algorithm process query statements and at least one query result corresponding to the sequence of algorithm process query statements, where the first storage statement based on query language is used to store the set of target tables based on the set of target distribution keys, each algorithm process query statement in the sequence of algorithmprocess query statements is used to query the set of tables and / or the set of historical query information.

[0028] In a possible design, each algorithm process query statement in the sequence of algorithm process query statements is a structured query language (SQL) query statement. The embodiments of the preset application do not limit the language of the query statement.

[0029] In a possible design, a first algorithm process query statement is used to process at least one of the following: at least one table with statistical information, the set of historical query information. Alternatively, the first algorithm process query statement is used to filter based on the value corresponding to at least one first attribute of a distribution key (e.g. estimated value of data skewness and / or query running time). The first algorithm process query statement belongs to the sequence of algorithm process query statements.

[0030] For example, the table with statistical information includes at least one of the following: cardinality of the table, a percentage of an undefined value in the target table, or the number of unique values in the target table. The table with statistical information is imported from DBMS technical table.

[0031] For example, the set of historical query information includes at least one of the following: query number of at least one historical query statement, query running time of at least one historical query statement, table being joined in at least one historical join query statement, cardinality of table in each historical join query statement before first join, etc.

[0032] In a possible design, the query result corresponding to the first algorithm process query statement is a new table, which is saved as a temporary result. For example, this new data table is used as inputs in future queries.

[0033] According to the method above, the cloud management platform provides interpretable data to the tenant, thereby the tenant understands the process of determining the set of target distribution keys, and the tenant directly uses the first storage statement to store the set of target tables based on the set of target distribution keys.

[0034] According to a second aspect, an embodiment of this application provides a device for obtaining a distribution key, where the device is applied to a cloud management platform, the cloud management platform manages an infrastructure used to provide cloud services, the infrastructure includes at least one data center, each data center in the at least one data centerincludes storage nodes, and the device includes: a transceiver unit is configured to receive a request from a tenant, where the request is used to obtain a set of target distribution keys, and each target distribution key in the set of target distribution keys is used to store a set of tables of the tenant in the storage nodes; and a processing unit is configured to: obtain a set of target tables and a set of candidate distribution keys based on the request and a set of historical query information, where the set of historical query information includes at least one historical join query statement, each historical join query statement of the at least one historical join query statement is used to query data in at least two target tables in the set of target tables, and a join key in each historical join query statement is a candidate distribution key for each target table corresponding to each historical join query statement; obtain a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule, where the first optimization rule includes a set of preset conditions corresponding to a set of first attributes of a distribution key, each preset condition in the set of preset conditions corresponds to a first attribute in the set of first attributes, and each first attribute corresponding to the set of first distribution keys satisfies the preset condition corresponding to the first attribute; and obtain the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys, where each target distribution key in the set of target distribution keys satisfies a first constraint condition, each target table in the set of target tables corresponds to a target distribution key in the set of target distribution keys, each target distribution key in the set of target distribution keys is a single-column distribution key or a multi-column distribution key, the network cost of each couple of first distribution keys in the set of first distribution keys is determined based on the network cost of at least one historical join query statement corresponding to the Couple of first distribution keys, and the couple of first distribution keys is determined based on the at least one historical join query statement.

[0035] In a possible design, when the target distribution key is a single-column distribution key, the target distribution key is determined based on a first distribution key. When the target distribution key is a multi-column distribution key, the target distribution key is determined based on multiple first distribution keys.

[0036] In a possible design, the set of first attributes includes: estimated value of data skewness and / or query running time, the estimated value of data skewness corresponding toeach candidate distribution key is used to estimate the unevenness of distribution on data when at least one target table corresponding to each candidate distribution key is stored in the storage nodes based on the candidate distribution key, and the query running time corresponding to the set of candidate distribution keys includes each time required to query according to each historical join query statement after storage of the set of target tables is simulated based on the set of candidate distribution keys.

[0037] In a possible design, the query running time corresponding to the set of candidate distribution keys is obtained by redistributing the set of target tables based on the set of candidate distribution keys. Alternatively, the query running time corresponding to the set of candidate distribution keys is obtained by simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the query running time corresponding to the set of candidate distribution keys is obtained by estimating manually by technician (e.g. DBMS planner).

[0038] In a possible design, the processing unit is configured to obtain the set of first distribution keys from the set of candidate distribution keys based on the estimated value of data skewness and / or query running time corresponding to the set of candidate distribution keys, where the estimated value of data skewness of each first distribution key is less than or equal to a first preset threshold, and / or, the query running time corresponding to the set of first distribution keys satisfies a first preset condition in the set of preset conditions.

[0039] In a possible design, the processing unit is configured to obtain a set of second distribution keys from the set of candidate distribution keys based on the estimated value of data skewness of each candidate distribution key, where the estimated value of data skewness of each second distribution key is less than or equal to a second preset threshold; and obtaining the set of first distribution keys from the set of second distribution keys based on the query running time corresponding to the set of second distribution keys, where the query running time corresponding to the set of first distribution keys satisfies the first preset condition.

[0040] In a possible design, the processing unit is configured to: when the query running time corresponding to the set of second distribution keys does not satisfy the first preset condition, obtain the set of first distribution keys from the set of second distribution keys, where the estimated value of data skewness of each first distribution key is less than or equal to a thirdpreset threshold, and the third preset threshold is less than the second preset threshold; or, when the query running time corresponding to the set of second distribution keys satisfies the first preset condition, determine the set of second distribution keys as the set of first distribution keys.

[0041] In a possible design, the first preset condition includes at least one of the following: a sum of query running time corresponding to the set of first distribution keys is less than or equal to a fourth preset threshold; or, a sum of query running time corresponding to the set of first distribution keys is less than or equal to a sum of query running time corresponding to a set of original distribution keys; or, a difference between a sum of query running time corresponding to the set of first distribution keys and a sum of query running time corresponding to a set of original distribution keys is less than or equal to a fifth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a sixth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a second query running time corresponding to a set of original distribution keys; or, a difference between a first query running time corresponding to the set of first distribution keys and a second query running time corresponding to a set of original distribution keys is less than or equal to a seventh preset threshold; where each original distribution key in the set of original distribution keys is a distribution key used by the tenant to store the set of tables, the first query running time is determined based on the query running time corresponding to the set of first distribution keys, and the second query running time is determined based on the query running time corresponding to a set of original distribution keys.

[0042] In a possible design, the sum of query running time corresponding to the set of first distribution keys is obtained by querying each historical join query statement after redistributing the set of target tables based on the set of first distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained by querying each historical join query statement after simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained by estimating manually by technician (e.g. DBMS planner).

[0043] In a possible design, the first constraint condition includes: the set of targetdistribution keys includes each target distribution key corresponding to each target table in the set of target tables, and each target distribution key is determined based on at least one first distribution key; the at least one first distribution key corresponding to a target distribution key corresponding to a first target table is determined based on at most one couple of first distribution keys corresponding to the first target table and each second target table in the set of second target tables, the first target table belongs to the set of target tables, each second target table and the first target table are associated target tables, and the associated target tables are tables queried through the same historical join query statement in the set of historical query information; and each first distribution key corresponding to the target distribution key is the distribution key in the set of first distribution keys that maximizes a value of ,where A is the set of target tables, a and b are different target tables in A, Wa'bis a network cost of a ith couple of the first distribution keys corresponding to a and b, Xa‘bis a first value of the ith couple of first distribution keys corresponding to a and b, Wa‘bis a non-negative number, 0 < Xa'b< 1 , 1 < i < , , lab is the number of couple of first distribution keys corresponding to a and b, and Iab is a positive integer.

[0044] In a possible design, the value of the Xa'bis any of the following: 0, 1, a decimal between 0 and 1.

[0045] In a possible design, the first value of each first distribution key corresponding to the target distribution key is greater than or equal to an eighth preset threshold, the first value of each first distribution key is a first value of a couple of first distribution keys which includes each first distribution key, and the processing unit is configured to provide the tenant with the set of target distribution keys, or, provide the tenant with the set of target distribution keys and the first value of each first distribution key corresponding to each target distribution key.

[0046] In a possible design, the processing unit is configured to obtain a statistic of each target table from a database of the tenant, where the statistic of each target table includes at least one of the following: a size of the target table, a percentage of an undefined value in the target table, or the number of unique values in the target table; and obtain the estimated value of data skewness of each candidate distribution key based on the statistic of each target table and theset of candidate distribution keys.

[0047] In a possible design, the transceiver unit is configured to provide the tenant with at least one of the following: at least one value corresponding to at least one first attribute of each first distribution key, the network cost of each couple of first distribution keys, a first storage statement, a sequence of algorithm process query statements and at least one query result corresponding to the sequence of algorithm process query statements, where the first storage statement based on query language is used to store the set of target tables based on the set of target distribution keys, each algorithm process query statement in the sequence of algorithm process query statements is used to query the set of tables and / or the set of historical query information.

[0048] According to a third aspect, an embodiment of this application provides a computing device cluster, including at least one computing device, wherein the computing device includes a processor and a memory coupled with the processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke and run the computer program stored in the memory, so that the computing device executes the method in any one of the first aspect or any possible design of the first aspect.

[0049] According to a fourth aspect, an embodiment of this application provides a computer program product including instructions, wherein when the computer program product is run on a server, the server is enabled to perform the method in any one of the first aspect or any possible design of the first aspect.

[0050] According to a fifth aspect, an embodiment of this application provides a computer readable storage medium including instructions, wherein when run on a server, the server is enabled to perform the method in any one of the first aspect or any possible design of the first aspect.DESCRIPTION OF DRAWINGS

[0051] FIG. 1 is a schematic block diagram of a system for obtaining a distribution key according to an embodiment of this application.

[0052] FIG. 2 shows an example of storing tables of a tenant.

[0053] FIG. 3 shows an example of storing tables of a tenant.

[0054] FIG. 4 is a schematic diagram of a method of obtaining a distribution key according to an embodiment of this application.

[0055] FIG. 5 is a schematic diagram of a method of obtaining a distribution key according to an embodiment of this application.

[0056] FIG. 6 is a schematic diagram of a device for obtaining a distribution key according to an embodiment of this application.

[0057] FIG. 7 is a schematic block diagram of a computing device according to an embodiment of this application.

[0058] FIG. 8 is a schematic block diagram of a computing device cluster according to an embodiment of this application.

[0059] FIG. 9 is a schematic block diagram of a computing device 700 A and a computing device 700B connected by a network according to an embodiment of this application.DESCRIPTION OF EMBODIMENTS

[0060] The following describes the technical solutions in the present application with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without making creative labor shall fall within the scope of protection of the present application.

[0061] The present application will present aspects, embodiments, or features around systems that include multiple devices, components, modules, etc. It should be understood and appreciated that the individual systems may include additional devices, components, modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in connection with the accompanying drawings. In addition, combinations of these options may be used.

[0062] In addition, in the embodiments of the present application, the word "exemplarily" and the phrase "as an example" are used to indicate for example, illustration or description. Anyembodiment or design solution described as "exemplarily" in this application should not be construed as being superior to or more advantageous than other embodiments or design solutions. Rather, the use of the word "example" is intended to present the concept in a specific manner.

[0063] The phrases "in some possible embodiments", "in some possible application scenarios", etc., appearing in various places in this description, do not necessarily refer to the same embodiments, but rather mean "one or more, but not all, embodiments" unless otherwise specifically emphasized. Unless otherwise specifically emphasized, the terms "including", "comprising", "having", and variations thereof all mean "including but not limited to".

[0064] In the present application, "at least one" refers to one or more, and "multiple" refers to two or more, "and / or", describing the association of the associated objects, indicates that three relationships can exist. For example, A and / or B can mean A alone, both A and B, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects are in an "or" relationship.

[0065] The application scenarios described in the present application embodiments are intended to illustrate the technical solutions of the present application embodiments more clearly and do not constitute a limitation to the technical solutions provided by the present application embodiments. It is known to those of ordinary skill in the art that the technical solutions provided by the present application embodiments are equally applicable to similar technical problems as the system architecture evolves and new application scenarios emerge.

[0066] The technical solution in the embodiments of the present application is applied to computing devices, such as servers, hosts, personal computers, laptops, desktop computers, etc. The server is a local server or a cloud server. This technical solution is applied to a cloud management platform when the server is a cloud server.

[0067] FIG. 1 is a schematic block diagram of a system 100 for obtaining a distribution key according to an embodiment of this application. In FIG. 1, the system 100 includes a cloud management platform 110. The cloud management platform 110 manages the infrastructure used to provide cloud services. This infrastructure is deployed in one or more data centers. This infrastructure includes a computing device cluster and / or a storage device cluster. The computing device cluster includes at least one computing device. When computing devices areincluded in the computing device cluster, they can be directly connected or connected with each other through a network. This network is, for example, a local area network or a wide area network. Each computing device in the computing device cluster can independently execute the method in the embodiments of the present application, or computing devices in the computing device cluster can jointly execute the method in the embodiments of the present application. The storage device cluster includes at least one storage device. When storage devices are included in the storage device cluster, they can be directly connected or connected with each other through a network. This network is, for example, a local area network or a wide area network. The storage device in the storage device cluster is either a centralized storage device or a distributed storage device, and embodiments of the present application are not limited to this.

[0068] As shown in FIG. 1, the cloud management platform 110 is used to manage a data center 120. The data center 120 includes storage nodes (such as a storage node 121 and a storage node 122), and each storage node is deployed on at least one storage device. For example, the data center 120 also includes a computing node 123. For example, the cloud management platform 110 is also used to manage a computing node 130, which is not located in the data center 120.

[0069] The storage node 121 and the storage node 122 are used to store the tenant's database, which includes at least one table of the tenant. The storage node 121 and the storage node 122 are also used to store data required for executing the method in the embodiments of the present application, such as at least one of the following: a request from the tenant, a set of historical query information, a set of candidate distribution keys, a set of first distribution keys, a set of target distribution keys, an estimated value of data skewness, a query running time, a network cost, etc.

[0070] The computing node 123 or the computing node 130 is used to receive a request from a tenant. The tenant is a public cloud tenant who has registered a public cloud account and purchased public cloud resources or services. The request is used to obtain a set of target distribution keys, and each target distribution key in the set of target distribution keys is used to store a set of tables of the tenant in the storage nodes.

[0071] The computing node 123 or the computing node 130 is also used to obtain a set oftarget tables and a set of candidate distribution keys based on the request and a set of historical query information. The set of historical query information includes at least one historical join query statement, and each historical join query statement of the at least one historical join query statement is used to query data in at least two target tables in the set of target tables. A join key in each historical join query statement is a candidate distribution key for each target table corresponding to each historical join query statement. In other words, each historical join query statement in the set of historical query information queries data in the set of tables through at least one join key. The join key in each historical join query statement is the candidate distribution key. The table queried in each historical join query statement is the target table.

[0072] The computing node 123 or the computing node 130 is also used to obtain a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule. The first optimization rule includes a set of preset conditions corresponding to a set of first attributes of a distribution key, each preset condition in the set of preset conditions corresponds to a first attribute in the set of first attributes, and each first attribute corresponding to the set of first distribution keys satisfies the preset condition corresponding to the first attribute.

[0073] In some embodiments, the set of first attributes includes: estimated value of data skewness and / or query running time. The estimated value of data skewness corresponding to each candidate distribution key is used to estimate the unevenness of distribution on data when at least one target table corresponding to each candidate distribution key is stored in the storage nodes based on the candidate distribution key. The query running time corresponding to the set of candidate distribution keys includes each time required to query according to each historical join query statement after storage of the set of target tables is simulated based on the set of candidate distribution keys.

[0074] When the set of first attributes includes the estimated value of data skewness, the preset condition for the estimated value of data skewness is that the estimated value of data skewness of the first distribution key is less than or equal to a first preset threshold. When the set of first attributes includes the query runtime, the preset condition for query running time is that the query running time corresponding to the set of first distribution keys satisfies a first preset condition in the set of preset conditions. In other words, the computing node 123 or the computing node 130 is also used to obtain the set of first distribution keys from the set ofcandidate distribution keys based on the estimated value of data skewness and / or query running time corresponding to the set of candidate distribution keys. The estimated value of data skewness of each first distribution key is less than or equal to a first preset threshold, and / or, the query running time corresponding to the set of first distribution keys satisfies a first preset condition in the set of preset conditions, as described in FIG. 4 or FIG. 5. The embodiments of the present application do not limit the value of the first preset threshold.

[0075] In some embodiments, the first optimization rule is defined by the cloud vendor or the tenant. In other words, the computing node 123 or the computing node 130 is also used to receive first configuration information from the tenant, which is used to indicate the first optimization rule.

[0076] The computing node 123 or the computing node 130 is also used to obtain the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys. Each target distribution key in the set of target distribution keys satisfies a first constraint condition. Each target table in the set of target tables corresponds to a target distribution key in the set of target distribution keys, and each target distribution key in the set of target distribution keys is a single-column distribution key or a multi-column distribution key. The network cost of each couple of first distribution keys in the set of first distribution keys is determined based on the network cost of at least one historical join query statement corresponding to the couple of first distribution keys, and the couple of first distribution keys is determined based on the at least one historical join query statement, as described in FIG. 4 or FIG. 5.

[0077] In some embodiments, when the target distribution key is a single-column distribution key, the target distribution key is determined based on a first distribution key. When the target distribution key is a multi-column distribution key, the target distribution key is determined based on multiple first distribution keys.

[0078] Optionally, when the computing node 123 or the computing node 130 obtains the set of target distribution keys, the first value of each couple of first distribution keys corresponding to each target distribution key is also obtained. The first value of each couple of first distribution keys is used to indicate the degree of recommendation for each couple of first distribution keys. The compute node 123 or the compute node 130 is also used to providetenants with the set of target distribution keys. Alternatively, the computing node 123 or the computing node 130 is also used to provide tenants with the set of target distribution keys and the first value of each couple of first distribution keys corresponding to each target distribution key.

[0079] In some embodiments, the first value of each couple of first distribution keys corresponding to each target distribution key is greater than or equal to an eighth preset threshold. The embodiments of the present application do not limit the value of the eighth preset threshold.

[0080] The system 100 in FIG. 1 removes the distribution key that does not satisfy the preset condition included in the first optimization rule from the set of candidate distribution keys, thereby reducing the number of potential distribution keys and improving the efficiency of obtaining the set of target distribution keys. Meanwhile, the first optimization rule can be flexibly adjusted, allowing the tenant or cloud vendor to flexibly exclude at least one inappropriate distribution key from the set of candidate distribution keys based on practical application scenarios. Moreover, the system 100 determines at least one target distribution key that satisfies the first constraint conditions from the set of first distribution keys based on the network cost corresponding to each couple of first distribution keys, and a single-column distribution key or a multi-column distribution key is determined for each target table, thereby minimizing the network cost of querying data after storing data based on the set of target distribution keys and improving the efficiency of querying data.

[0081] FIG. 2 shows an example of storing the tables of the tenant. As shown in FIG. 2, for example, the tenant's database includes table 1 and table 2. Table 1 and table 2 are stored in column storage form in a storage node 210 and a storage node 220, and the storage node 210 and the storage node 220 are deployed on different storage devices. Table 1 includes a column with a data attribute named "name", which includes at least 6 data, namely: name-1, name-2, name-3, name-1, name-1, name-4. Table 2 includes a column with the data attribute named "name", which contains at least 6 data, namely: name-1, name- 1, name-2, name-3, name-4, and name-1. In FIG. 2, the first three data in the column with the data attribute named "name" in table 1 are stored in the storage node 210, and the last three data in the column with the data attribute named "name" in table 1 are stored in the storage node 220. The first three data in thecolumn with the data attribute named "name" in table 2 are stored in the storage node 210, and the last three data in the column with the data attribute named "name" in table 2 are stored in the storage node 220. Data-2, located in the same row as data-1 in the column with the data attribute named "name" in table 1, is stored in the same storage node as data-1. For example, the data in table 1 located on the same row as the first name-1 is stored in the storage node 210.

[0082] When the tenant needs to query the relevant data with name-1 in table 1 and table 2, the tenant uses a join query statement with the data attribute named "name" as the join key and "name" as name-1. When the tenant uses the join query statement for querying, if the computing node and the storage node 210 are deployed on the same device, the computing node needs to access the storage node 220 and obtain relevant data with the name-1 in table 1 and table 2 from the storage node 220 (such as some or all of the data in the same row as name-1 in table 1 and table 2). That is, the network cost corresponding to the join query statement is the network cost of transmitting the relevant data with name-1 in table 1 or table 2. If the amount of relevant data is large, the network cost corresponding to the join query statement is higher, resulting in a longer time to obtain query results, and the efficiency of querying is lower.

[0083] In some embodiments, the network cost is determined based on the number of bytes of transmitted data.

[0084] FIG. 3 shows another example of storing the tables of the tenant. As shown in FIG. 3, for example, the tenant's database includes table 1 and table 2. Table 1 and table 2 are stored in column storage form in a storage node 210 and a storage node 220, and the storage node 210 and the storage node 220 are deployed on different storage devices. Table 1 includes a column with a data attribute named "name", which includes at least 6 data, namely: name-1, name-2, name-3, name-1, name-1, name-4. Table 2 includes a column with the data attribute named "name", which contains at least 6 data, namely: name-1, name-1, name-2, name-3, name-4, and name-1. In FIG. 3, the data with name-1 in the column with the data attribute named "name" in table 1 is stored in the storage node 210, while other data in the column with the data attribute named "name" in table 1 is stored in the storage node 220. The data with name-1 in the column with the data attribute named "name" in table 2 is stored in storage node 210, while other data in the column with the data attribute named "name" in table 2 is stored in storage node 220. In other words, in FIG. 3, table 1 and table 2 are stored with "name" as a distribution key, so thatthe same data (or data with the same hash value) in the columns with "name" as the data attribute in table 1 and table 2 are stored in the same storage node. Data-2, located in the same row as a data-1 in the column with the data attribute named "name" in table 1, is stored in the same storage node as the data-1. For example, the data in table 1 located on the same row as the name-1 is stored in the storage node 210.

[0085] When the tenant needs to query the relevant data with name-1 in table 1 and table 2, the tenant uses a join query statement with the data attribute named "name" as the join key and "name" as name-1. When the tenant uses the join query statement for querying, since the data with name-1 in table 1 and table 2 are stored in the storage node 210, the computing node does not need to access other storage nodes (such as the storage node 220) to return the query results to the tenant. In other words, when table 1 and table 2 is stored based on the distribution key "name" and the join query statement with the join key "name" is used for querying, the network cost corresponding to the join query statement is lower, the query running time corresponding to the join query statement is shorter, and the efficiency of querying is higher.

[0086] According to the descriptions in FIG. 2 and FIG. 3, using the appropriate distribution key for column storage in the table saves the network cost and the query running time, thereby improving the efficiency of querying. However, when the tenant has a large number of tables and at least one table contains a large number of data attributes, it is difficult to manually determine the appropriate distribution key.

[0087] FIG. 4 is a schematic diagram of a method of obtaining a distribution key according to an embodiment of this application. The method in FIG. 4 is applied to the cloud management platform in FIG. 1. The method in FIG. 4 includes the following steps.

[0088] Step 410: receiving a request from a tenant.

[0089] The cloud management platform receives the request from the tenant. The request is used to obtain a set of target distribution keys, and each target distribution key in the set of target distribution keys is used to store a set of tables of the tenant in the storage nodes. The tenant is a public cloud tenant who has registered a public cloud account and purchased public cloud resources or services.

[0090] In some embodiments, the request includes first indication information, and the first indication information is used to indicate whether to allow a table to have a single-columndistribution key or a multi-column distribution key.

[0091] Optionally, the cloud management platform provides the tenant with a first graphical interface, allowing for uploading or selecting operations on the first graphical interface, thereby obtaining the request of the tenant. The embodiments of the present application do not limit the specific manifestation of the first graphical interface.

[0092] Step 420: obtaining a set of target tables and a set of candidate distribution keys based on the request and a set of historical query information.

[0093] After receiving the request from the tenant, the cloud management platform obtains the set of historical query information based on the request. The set of historical query information includes at least one historical join query statement, and each historical join query statement of the at least one historical join query statement is used to query data in at least two target tables in the set of target tables.

[0094] Optionally, the set of historical query information is uploaded by the tenant. Alternatively, the set of historical query information includes the query information of the tenant when querying the set of tables during historical time.

[0095] Optionally, the cloud management platform provides the tenant with a second graphical interface, allowing for uploading or selecting operations on the second graphical interface, thereby obtaining the set of historical query information. The embodiments of the present application do hot limit the specific manifestation of the second graphical interface.

[0096] In some embodiments, the request includes second indication information, and the second indication information is used to indicate at least one table of the tenant and / or a first time period corresponding to the set of historical query information. The first time period is any time period within the historical time. In other words, the tenant indicates in the request: determining the set of target distribution keys for which tables, and / or, determining the set of target distribution keys corresponding to the set of tables based on the set of historical query information in which historical time.

[0097] In some embodiments, when the set of tables is stored in column storage form, the set of tables corresponds to a set of original distribution keys. Each original distribution key in the set of original distribution keys is used to store the set of tables of the tenant. The set of original distribution keys is determined by the tenant, or, the set of original distribution keys isany data attribute in at least one data attribute corresponding to the set of tables. In other words, the set of original distribution keys is not the set of target distribution keys determined according to the method in the embodiments of the present application.

[0098] After obtaining the set of historical query information, the cloud management platform obtains a set of target tables and a set of candidate distribution keys based on the set of historical query information. A join key in each historical join query statement is a candidate distribution key for each target table corresponding to each historical join query statement. In other words, each historical join query statement in the set of historical query information queries data in the set of tables through at least one join key. The join key in each historical join query statement is the candidate distribution key. The table queried in each historical join query statement is the target table.

[0099] In some embodiments, the set of historical query information includes at least one of the following: the network cost of each historical join query statement, and the query running time of each historical join query statement.

[0100] In some embodiments, the network cost is determined based on the number of bytes of transmitted data. The network cost of each historical join query statement is determined based on the number of bytes of data that need to be transmitted during the query for each historical join query statement.

[0101] For example, if a first historical join query statement queries the data in a first table and a second table through a first join key, the first table and the second table both include the first join key, the first table and the second table are the target tables, and the first join key is the candidate distribution key. A couple of candidate distribution keys corresponding to the first historical join query statement is (the first join key in the first table, the first join key in the second table). The network cost of the couple of candidate distribution keys is the network cost of the first historical join query statement. At least one candidate distribution key corresponding to the first table includes the first join key, that is, the first table has a corresponding relationship with the first join key. At least one candidate distribution key corresponding to the second table includes the first join key, that is, the second table has a corresponding relationship with the first join key. At least one couple of candidate distribution keys corresponding to the first table and the second table includes this couple of candidate distribution keys, that is, the first tableand the second table have a corresponding relationship with this couple of candidate distribution keys.

[0102] For example, if a second historical join query statement queries the data in a first table and a second table through a first join key and a second join key, the first table includes the first join key, the second table includes the second join key, the first table and the second table are the target tables, and the first join key and the second join key are the candidate distribution keys. A couple of candidate distribution keys corresponding to the second historical join query statement is (the first join key, the second join key). The network cost of this couple of candidate distribution keys is the network cost of the second historical join query statement.

[0103] Optionally, the cloud management platform obtains the identification of the set of target tables based on the set of historical query information, and obtains the metadata information of the set of target tables from the tenant's database based on the identification of each target table. Among them, the identification of the target data table includes the name of the target table and / or the index of the target table in the tenant's database. The metadata information of the target data table includes at least one of the following: the cardinality of the target table, the size of the target table, the type of the data in the target table, the data attribute of the column in the target table (e.g. the name of the column), the number of rows, the percentage of an undefined value in each column, the number of unique values in each column and etc. Among them, the undefined value (such as NAN or null) indicates that there is not a specific value at that location. The number of unique values in each column is the number of values obtained by deduplicating the data in this column.

[0104] Step 430: obtaining a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule.

[0105] The first optimization rule includes a set of preset conditions corresponding to a set of first attributes of a distribution key, each preset condition in the set of preset conditions corresponds to a first attribute in the set of first attributes, and each first attribute corresponding to the set of first distribution keys satisfies the preset condition corresponding to the first attribute. The embodiments of the present application do not limit the type of the first attribute in the set of first attributes.

[0106] Optionally, the first optimization rule is defined by the cloud vendor or the tenant.

[0107] In some embodiments, the cloud management platform receives the first configuration information from the tenant, which is used to indicate the first optimization rule.

[0108] Optionally, the cloud management platform provides the tenant with a third graphical interface, allowing for uploading or selecting operations on the third graphical interface, thereby obtaining the first configuration information. The embodiments of the present application do not limit the specific manifestation of the third graphical interface.

[0109] Optionally, the set of first attributes includes: estimated value of data skewness and / or query running time. The estimated value of data skewness corresponding to each candidate distribution key is used to estimate the unevenness of distribution on data when at least one target table corresponding to each candidate distribution key is stored in the storage nodes based on this candidate distribution key. The query running time corresponding to the set of candidate distribution keys includes each time required to query according to each historical join query statement after storage of the set of target tables is simulated based on the set of candidate distribution keys.

[0110] In some embodiments, the query running time corresponding to the set of candidate distribution keys is obtained by redistributing the set of target tables based on the set of candidate distribution keys. Alternatively, the query running time corresponding to the set of candidate distribution keys is obtained by simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the query running time corresponding to the set of candidate distribution keys is obtained by estimating manually by technician (e.g. DBMS planner).

[0111] Optionally, the cloud management platform obtains the set of first distribution keys from the set of candidate distribution keys based on the estimated value of data skewness and / or query running time corresponding to the set of candidate distribution keys.

[0112] Optionally, the cloud management platform obtains the set of first distribution keys from the set of candidate distribution keys based on the estimated value of data skewness corresponding to the set of candidate distribution keys. The estimated value of data skewness of each first distribution key is less than or equal to a first preset threshold. The embodiments of the present application do not limit the specific value of the first preset threshold. The estimated value of data skewness of the distribution key is used to estimate the unevenness ofdistribution on data when at least one target table corresponding to each candidate distribution key is stored in the storage nodes based on this distribution key.

[0113] In some embodiments, the estimated value of data skewness of each candidate distribution key is obtained based on the statistic of each target table. The statistic of each target table includes at least one of the following: a size of the target table, a percentage of an undefined value in the target table, or the number of unique values in the target table. The calculation method for estimated value of data skewness is described in step 510.

[0114] Optionally, the cloud management platform obtains the set of first distribution keys from the set of candidate distribution keys based on the query running time corresponding to the set of candidate distribution keys. The query running time corresponding to the set of first distribution keys satisfies a first preset condition in the set of preset conditions. The query running time corresponding to the set of candidate distribution keys includes each time required to query according to each historical join query statement after storage of the set of target tables is simulated based on the set of candidate distribution keys.

[0115] In some embodiments, the first preset condition includes at least one of the following: a sum of query running time corresponding to the set of first distribution keys is less than or equal to a fourth preset threshold; or, a sum of query running time corresponding to the set of first distribution keys is less than or equal to a sum of query running time corresponding to a set of original distribution keys; or, a difference between a sum of query running time corresponding to the set of first distribution keys and a sum of query running time corresponding to a set of original distribution keys is less than or equal to a fifth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a sixth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a second query running time corresponding to a set of original distribution keys; or, a difference between a first query running time corresponding to the set of first distribution keys and a second query running time corresponding to a set of original distribution keys is less than or equal to a seventh preset threshold. Each original distribution key in the set of original distribution keys is a distribution key used by the tenant to store the set of tables, the first query running time is determined based on the query running time corresponding to the set of first distribution keys, and the second query running time isdetermined based on the query running time corresponding to a set of original distribution keys.

[0116] Among them, the sum of query running time corresponding to the set of first distribution keys is the sum of time required to query each historical join query statement in the set of historical query information after storage of the set of target tables is simulated based on the set of first distribution keys. The sum of query running time corresponding to the set of original distribution keys is the sum of time required to query each historical join query statement in the set of historical query information after storage of the set of target tables is simulated based on the set of original distribution keys. The first query running time is the query running time of the first historical join query statement after storage of the set of target tables is simulated based on the set of first distribution keys. The second query running time is the query running time of the first historical join query statement after storage of the set of target tables is simulated based on the set of original distribution keys. The first historical join query statement is any historical join query statement in the set of historical query information. Alternatively, the first historical join query statement is the historical join query statement with the longest (or shortest) query running time in the set of historical query information after storage of the set of target tables is simulated based on the set of original distribution keys. Alternatively, the first historical join query statement is the historical join query statement with the longest (or shortest) query running time in the set of historical query information after storage of the set of target tables is simulated based on the set of first distribution keys. Alternatively, the first query running time is the average query running time of each historical join query statement after storage of the set of target tables is simulated based on the set of first distribution keys, and the second query running time is the average query running time of each historical join query statement after storage of the set of target tables is simulated based on the set of original distribution keys. The specific values of the fourth preset threshold, the fifth preset threshold, the sixth preset threshold, and the seventh preset threshold are not limited in the embodiments of the present application.

[0117] In some embodiments, the sum of query running time corresponding to the set of first distribution keys is obtained by querying each historical join query statement after redistributing the set of target tables based on the set of first distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained byquerying each historical join query statement after simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained by estimating manually by technician (e.g. DBMS planner).

[0118] In some embodiments, the first query running time is obtained by querying the first historical join query statement after redistributing the set of target tables based on the set of first distribution keys. Alternatively, the first query running time is obtained by querying the first historical join query statement after simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the first query running time is obtained by estimating manually by technician (e.g. DBMS planner).

[0119] Optionally, the cloud management platform obtains a set of second distribution keys from the set of candidate distribution keys based on the estimated value of data skewness of each candidate distribution key. The estimated value of data skewness of each second distribution key is less than or equal to a second preset threshold. The embodiments of the present application do not limit the value of the second preset threshold. The first preset threshold and the second preset threshold are the same or different. And, the cloud management platform obtains the set of first distribution keys from the set of second distribution keys based on the query running time corresponding to the set of second distribution keys. The query running time corresponding to the set of first distribution keys satisfies the first preset condition. This method is described in FIG. 5.

[0120] Step 440: obtaining the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys.

[0121] Each target distribution key in the set of target distribution keys satisfies a first constraint condition. Each target table in the set of target tables corresponds to a target distribution key in the set of target distribution keys, and each target distribution key in the set of target distribution keys is a single-column distribution key or a multi-column distribution key. The network cost of each couple of first distribution keys is determined based on the network cost of at least one historical join query statement corresponding to the couple of first distribution keys, and the couple of first distribution keys is determined based on the at least one historical join query statement.

[0122] In some embodiments, when the target distribution key is a single-column distribution key, the target distribution key is determined based on a first distribution key. When the target distribution key is a multi-column distribution key, the target distribution key is determined based on multiple first distribution keys.

[0123] For example, storing a first table based on a first single-column distribution key includes: hashing the data in a first column of the first table and storing data with the same hash value in the same storage node. The first single-column distribution key corresponds to the first column in the first table, and the first table includes at least one column. In other words, each row in the first table corresponds to a hash value, which is determined by hashing the data in the first column of each row.

[0124] For example, storing the first table based on a first multi-column distribution key includes: hashing the data in the first column and second column of the first table, and storing data with the same hash value in the same storage node. The first multi-column distribution key corresponds to the first column and second column in the first table, and the first table includes multiple columns. In other words, each row in the first table corresponds to a hash value, which is determined by hashing the data in the first column and second column of each row.

[0125] When the set of historical query information includes the network cost of each historical join query statement, the network cost of each couple of first distribution keys is determined based on at least one historical join query statement corresponding to each couple of first distribution keys. The historical join query statement corresponding to the couple of first distribution keys is a historical join query statement that uses the couple of first distribution keys as the join keys for querying. When the couple of first distribution keys corresponds to a historical join query statement, the network cost of the couple of first distribution keys is the network cost of this historical join query statement. When the couple of first distribution keys corresponds to multiple historical join query statements, the network cost of the couple of first distribution keys is the sum of the network cost of each historical join query statement.

[0126] In some embodiments, the first constraint condition includes: the set of target distribution keys includes each target distribution key corresponding to each target table in the set of target tables, and each target distribution key is determined based on at least one first distribution key; the at least one first distribution key corresponding to a target distribution keycorresponding to a first target table is determined based on at most one couple of first distribution keys corresponding to the first target table and each second target table in the set of second target tables, the first target table belongs to the set of target tables, each second target table and the first target table are associated target tables, and the associated target tables are tables queried through the same historical join query statement in the set of historical query information; and each first distribution key corresponding to the target distribution key is the distribution key in the set of first distribution keys that maximizes a value of ^X‘„ •A is the set of target tables (e.g. set A), a and b are different target tables in A (e.g. the table a and the table b) Wa'bis a network cost of a ith couple of first distribution keys corresponding to a and b, Xa‘bis a first value of the ith couple of first distribution keys corresponding to a and b, Wa’bis a non-negative number, 0 < Xa‘b< 1 , 1 < i < / ^ , Iab is the number of couple of first distribution keys corresponding to a and b, and lab is a positive integer. In other words, the value of the Xa'bis any of the following: 0, 1 , a decimal between 0 and 1.

[0127] In some embodiments, the higher X‘ih, the higher the recommendation level of the ith couple of first distribution keys. In other words, if Xa'bis higher, it indicates that after storing the set of target tables based on the ith couple of first distribution keys, the network cost for querying is lower and the efficiency of querying is higher. Correspondingly, the lower Xa'b, the lower the recommendation level of the ith couple of first distribution keys. In other words, if Xa'bis lower, it indicates that after storing the set of target tables based on the ith couple of first distribution keys, the network cost for querying is higher and the efficiency of querying is lower. In a word, Xa'bis used to indicate the degree of recommendation for the ith couple of first distribution keys corresponding to a and b.

[0128] In some embodiments, the first value of each first distribution key corresponding to the target distribution key is greater than or equal to an eighth preset threshold, and the first value of each first distribution key is a first value of a couple of first distribution keys which includes this first distribution key. The embodiments of the present application do not limit thevalue of the eighth preset threshold.

[0129] Optionally, the cloud management platform provides the tenant with the set of target distribution keys, or, provides the tenant with the set of target distribution keys and the first value of each couple of first distribution keys corresponding to each target distribution key.

[0130] According to the method in FIG. 4, the distribution key with higher estimated value of data skewness is removed from the set of candidate distribution keys, thereby avoiding tenants from storing the set of target tables based on the distribution key with higher estimated value of data skewness, and improving the efficiency of querying. Moreover, according to this method, the query running time corresponding to the set of candidate distribution keys is simulated to avoid obtaining the distribution key with a longer query running time, thereby improving the efficiency of querying after storing tables based on the set of target distribution keys. Meanwhile, according to this method, the set of target distribution keys that satisfies the first constraint condition is obtained from the set of first distribution keys based on the network cost of each couple of first distribution keys, and a single-column distribution key or a multi- column distribution key is determined for each target table, thereby minimizing the network cost of querying after storing tables based on the set of target distribution keys and improving the efficiency of querying data.

[0131] According to the method in FIG. 4, the first optimization rule is added during production testing. Thereby, this method is more robust than models or algorithms based on mathematical modeling. And, the tenant can see temporal results on almost every step, which helps to understand the reason behind distribution keys suggestion, and, in some specific cases, to decide to not apply suggested changes. This method makes the optimization part of the algorithm interpretable. Instead of only one suggestion for each table, variables between 0 and 1 are returned, responding to estimate the usefulness of the column as a distribution key. Meanwhile, multiple database setups use multi-column distribution keys as a useful alternative to Round-Robbin distribution. Moreover, complex distribution keys, chosen as join keys, can reduce network costs as one-column distribution keys do.

[0132] FIG. 5 is a schematic diagram of a method of obtaining a distribution key according to an embodiment of this application. The method in FIG. 5 is applied to the cloud management platform in FIG. 1. The method in FIG. 5 includes the following steps.

[0133] Step 510: obtaining a set of second distribution keys from the set of candidate distribution keys based on the estimated value of data skewness of each candidate distribution key.

[0134] Optionally, the cloud management platform obtains the estimated value of data skewness of each candidate distribution key before step 510.

[0135] In some embodiments, the cloud management platform obtains the set of target tables corresponding to each candidate distribution key, and obtains the percentage of undefined value in the column of each target table corresponding to the candidate distribution key and the number of unique values in the column of each target table corresponding to the candidate distribution key. The cloud management platform obtains the estimated value of data skewness of the candidate distribution key based on the percentage of the undefined value and the number of unique values. For example, the cloud management platform obtains a number of defined values based on the percentage of the undefined value in the column of each target table corresponding to the candidate distribution key. The cloud management platform obtains the estimated value of data skewness of the candidate distribution key based on the number of defined values and the number of unique values, e.g. the estimated value of data skewness = the number of defined values / the number of unique values.

[0136] For example, if the target table corresponding to the first candidate distribution key includes table 1 and table 2. In other words, table 1 includes a column corresponding to the first candidate distribution key, and table 2 includes a column corresponding to the first candidate distribution key. The number of values in the column corresponding to the first candidate distribution key in table 1 is 110, the percentage of the undefined value in the column corresponding to the first candidate distribution key in table 1 is 10%, and the number of defined values in the column corresponding to the first candidate distribution key in table 1 is (110- 110*10%) = 100. The number of values in the column corresponding to the first candidate distribution key in table 2 is 100, the percentage of the undefined value in the column corresponding to the first candidate distribution key in table 2 is 20%, and the number of defined values in the column corresponding to the first candidate distribution key in table 2 is (100- 100*20%) = 80. The number of unique values in the column corresponding to the first candidate distribution key in table 1 and table 2 is 10, and the estimated value of data skewness of the firstcandidate distribution key is (100+80) / l 0=18.

[0137] In some embodiments, the cloud management platform obtains the set of target tables corresponding to each candidate distribution key, and obtains the number of unique values in the column of each target table corresponding to the candidate distribution key. The cloud management platform obtains the estimated value of data skewness of the candidate distribution key based on: (numl - num2) / numl, where numl is a maximum number of data corresponding to the unique value, and num2 is an average number of data corresponding to the unique value.

[0138] For example, if the target table corresponding to the first candidate distribution key includes table 1 and table 2. In other words, table 1 includes a column corresponding to the first candidate distribution key, and table 2 includes a column corresponding to the first candidate distribution key. The number of unique values in the column corresponding to the first candidate distribution key in table 1 and table 2 is 5, and the unique values are: vl, v2, v3, v4 and v5. The number of data corresponding to vl in table 1 and table 2 is 50, the number of data corresponding to v2 in table 1 and table 2 is 10, the number of data corresponding to v3 in table 1 and table 2 is 8, the number of data corresponding to v4 in table 1 and table 2 is 2, and the number of data corresponding to v5 in table 1 and table 2 is 10. In other words, the maximum number of data corresponding to unique values (vl - v5) is 50, the average number of data corresponding to unique values (vl - v5) is 16, and the estimated value of data skewness of the first candidate distribution key is (50-16) / 50=0.68.

[0139] Optionally, after obtaining the estimated value of data skewness of each candidate distribution key, the cloud management platform obtains the second distribution key with the estimated value of data skewness less than or equal to the second preset threshold, thereby obtaining the set of second distribution keys. The embodiments of the present application do not limit the specific value of the second preset threshold. The second preset threshold and the first preset threshold are the same or different.

[0140] Step 520: obtaining the set of first distribution keys from the set of second distribution keys based on the query running time corresponding to the set of second distribution keys.

[0141] After obtaining the set of second distribution keys, the cloud management platformsimulates the storage of the set of target tables based on the set of second distribution keys. The cloud management platform queries the set of target tables based on each historical join query statement in the set of historical query information, in order to obtain the query running time of each historical join query statement. The query running time of each historical join query statement includes the time period from the start of the query for each historical join query statement to the end of the time when the query result is obtained. The query running time corresponding to the set of second distribution keys includes the query running time of each historical join query statement in the set of historical query information.

[0142] The cloud management platform obtains the set of first distribution keys from the set of second distribution keys based on the first preset condition and the query running time corresponding to the set of second distribution keys. The query running time corresponding to the set of first distribution keys satisfies the first preset condition. Specifically, when the query running time corresponding to the set of second distribution keys does not satisfy the first preset condition, the cloud management platform obtains the set of first distribution keys from the set of second distribution keys; when the query running time corresponding to the set of second distribution keys satisfies the first preset condition, the cloud management platform determines the set of second distribution keys as the set of first distribution keys.

[0143] In some embodiments, a second attribute in the set of first attributes corresponding to the first distribution key satisfies a second preset condition in the set of preset conditions. In other words, the cloud management platform obtains the second attribute corresponding to each second distribution key, and obtains the first distribution key with the second attribute satisfying the second preset condition. The second attribute belongs to the set of first attributes.

[0144] For example, the second attribute is the estimated value of data skewness. The second preset condition is that the estimated value of data skewness of each first distribution key is less than or equal to a third preset threshold. The third preset threshold is less than the second preset threshold. The embodiments of the present application do not limit the specific value of the third preset threshold. In other words, when the query running time corresponding to the set of second distribution keys does not satisfy the first preset condition, the cloud management platform obtains the first distribution key with the estimated value of data skewness is less than or equal to the third preset threshold from the set of second distributionkeys.

[0145] In some embodiments, the first preset condition includes at least one of the following: a sum of query running time corresponding to the set of first distribution keys is less than or equal to a fourth preset threshold; or, a sum of query running time corresponding to the set of first distribution keys is less than or equal to a sum of query running time corresponding to a set of original distribution keys; or, a difference between a sum of query running time corresponding to the set of first distribution keys and a sum of query running time corresponding to a set of original distribution keys is less than or equal to a fifth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a sixth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a second query running time corresponding to a set of original distribution keys; or, a difference between a first query running time corresponding to the set of first distribution keys and a second query running time corresponding to a set of original distribution keys is less than or equal to a seventh preset threshold. The sum of query running time corresponding to the set of first distribution keys is the sum of time required to query each historical join query statement in the set of historical query information after storage of the set of target tables is simulated based on the set of first distribution keys. The sum of query running time corresponding to the set of original distribution keys is the sum of time required to query each historical join query statement in the set of historical query information after storage of the set of target tables is simulated based on the set of original distribution keys. The specific values of the fourth preset threshold, the fifth preset threshold, the sixth preset threshold, and the seventh preset threshold are not limited in the embodiments of the present application.

[0146] Optionally, the set of historical query information includes the query running time of each historical join query statement when the set of target tables is stored according to the set of original distribution keys.

[0147] In some embodiments, the cloud management platform obtains the set of first distribution keys based on the first preset condition and the query running time corresponding to the set of second distribution keys. Specifically, when the sum of query running time corresponding to the set of second distribution keys is less than or equal to the fourth presetthreshold, and / or, when the first query running time corresponding to the set of second distribution keys is less than or equal to the sixth preset threshold, the cloud management platform determines each second distribution key as the first distribution key. When the sum of query running time corresponding to the set of second distribution keys is greater than the fourth preset threshold, and / or, when the first query running time corresponding to the set of second distribution keys is greater than the sixth preset threshold, the cloud management platform obtains the set of first distribution keys from the set of second distribution keys. The first query running time is the query running time of the first historical join query statement after storage of the set of target tables is simulated based on the set of first distribution keys. The first historical join query statement is any historical join query statement in the set of historical query information. Alternatively, the first historical join query statement is the historical join query statement with the longest (or shortest) query running time in the set of historical query information after storage of the set of target tables is simulated based on the set of original distribution keys. Alternatively, the first historical join query statement is the historical join query statement with the longest (or shortest) query running time in the set of historical query information after storage of the set of target tables is simulated based on the set of first distribution keys. Alternatively, the first query running time is the average query running time of each historical join query statement after storage of the set of target tables is simulated based on the set of first distribution keys.

[0148] In some embodiments, the cloud management platform obtains the set of first distribution keys based on the first preset condition, the query running time corresponding to the set of second distribution keys, and the query running time corresponding to the set of original distribution keys. Specifically, when the sum of query running time corresponding to the set of first distribution keys is less than or equal to the sum of query running time corresponding to the set of original distribution keys, or, when the first query running time corresponding to the set of first distribution keys is less than or equal to the second query running time corresponding to the set of original distribution keys, or, when the difference between the sum of query running time corresponding to the set of first distribution keys and the sum of query running time corresponding to the set of original distribution keys is less than or equal to the fifth preset threshold, or, when the difference between a first query running timecorresponding to the set of first distribution keys and a second query running time corresponding to a set of original distribution keys is less than or equal to the seventh preset threshold, the cloud management platform determines each second distribution key as the first distribution key. When the sum of query running time corresponding to the set of first distribution keys is greater than the sum of query running time corresponding to the set of original distribution keys, or, when the first query running time corresponding to the set of first distribution keys is greater than the second query running time corresponding to the set of original distribution keys, or, when the difference between the sum of query running time corresponding to the set of first distribution keys and the sum of query running time corresponding to the set of original distribution keys is greater than the fifth preset threshold, or, when the difference between a first query running time corresponding to the set of first distribution keys and a second query running time corresponding to a set of original distribution keys is greater than the seventh preset threshold, the cloud management platform obtains the set of first distribution keys from the set of second distribution keys.

[0149] Among them, the first query running time is the query running time of the first historical join query statement after storage of the set of target tables is simulated based on the set of first distribution keys. The second query running time is the query running time of the first historical join query statement after storage of the set of target tables is simulated based on the set of original distribution keys. The first historical join query statement is any historical join query statement in the set of historical query information. Alternatively, the first historical join query statement is the historical join query statement with the longest (or shortest) query running time in the set of historical query information after storage of the set of target tables is simulated based on the set of original distribution keys. Alternatively, the first historical join query statement is the historical join query statement with the longest (or shortest) query running time in the set of historical query information after storage of the set of target tables is simulated based on the set of first distribution keys. Alternatively, the first query running time is the average query running time of each historical join query statement after storage of the set of target tables is simulated based on the set of first distribution keys, and the second query running time is the average query running time of each historical join query statement after storage of the set of target tables is simulated based on the set of original distribution keys.

[0150] In some embodiments, the sum of query running time corresponding to the set of first distribution keys is obtained by querying each historical join query statement after redistributing the set of target tables based on the set of first distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained by querying each historical join query statement after simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the sum of query running time corresponding to the set of first distribution keys is obtained by estimating manually by technician (e.g. DBMS planner).

[0151] In some embodiments, the first query running time is obtained by querying the first historical join query statement after redistributing the set of target tables based on the set of first distribution keys. Alternatively, the first query running time is obtained by querying the first historical join query statement after simulating the distribution of the set of target tables based on the set of candidate distribution keys. Alternatively, the first query running time is obtained by estimating manually by technician (e.g. DBMS planner).

[0152] Step 530: obtaining the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys.

[0153] Each target distribution key in the set of target distribution keys satisfies a first constraint condition. Each target table in the set of target tables corresponds to a target distribution key in the set of target distribution keys, and each target distribution key in the set of target distribution keys is a single-column distribution key or a multi-column distribution key. The network cost of each couple of first distribution keys in the set of first distribution keys is determined based on the network cost of at least one historical join query statement corresponding to the couple of first distribution keys, and the couple of first distribution keys is determined based on the at least one historical join query statement. The single-column distribution key or multi-column distribution key is described in step 440.

[0154] For example, when the couple of first distribution keys corresponds to a historical join query statement, the network cost of the couple of first distribution keys is the network cost of this historical join query statement. When the couple of first distribution keys corresponds to multiple historical join query statements, the network cost of the couple of first distribution keys is the sum of the network cost of each historical join query statement.

[0155] In some embodiments, the first constraint condition includes: the set of target distribution keys includes each target distribution key corresponding to each target table in the set of target tables, and each target distribution key is determined based on at least one first distribution key; the at least one first distribution key corresponding to a target distribution key corresponding to a first target table is determined based on at most one couple of first distribution keys corresponding to the first target table and each second target table in the set of second target tables, the first target table belongs to the set of target tables, each second target table and the first target table are associated target tables, and the associated target tables are tables queried through the same historical join query statement in the set of historical query information; and each first distribution key corresponding to the target distribution key is the distribution key in the set of first distribution keys that maximizes a value of .A is the set of target tables (e.g. set A), a and b are different target tables in A (e.g. the table a and the table b), Wbis a network cost of a ith couple of first distribution keys corresponding to a and b, Xa‘bis a first value of the ith couple of first distribution keys corresponding to a and b, Wa'bis a non-negative number, 0 < Xa‘b< 1 , 1 < i < I* , Iab is the number of couple of first distribution keys corresponding to a and b, and Iab is a positive integer. In other words, the value of the Xa'bis any of the following: 0, 1, a decimal between 0 and 1.

[0156] In some embodiments, the first value of each first distribution key corresponding to the target distribution key is greater than or equal to an eighth preset threshold, and the first value of each first distribution key is a first value of a couple of first distribution keys which includes this first distribution key. The embodiments of the present application do not limit the value of the eighth preset threshold.

[0157] Optionally, the cloud management platform obtains at least one couple of first distribution keys based on the network cost of each couple of first distribution keys, the set of target tables, and the set of first distribution keys. Each couple of first distribution keys in the at least one couple of first distribution keys satisfies a first constraint condition. In other words, the cloud management platform obtains the couple of first distribution keys corresponding to each two associated target tables in the set of target tables, so that the sum of each product ofthe network cost of each couple of first distribution keys and the first value of this couple of first distribution keys are maximized, thereby obtaining the set of target distribution keys. The first value of each couple of first distribution keys is greater than or equal to 0, and the first value of each couple of first distribution keys is less than or equal to 1. That is, the range of values for the first value of each couple of first distribution keys is [0,1].

[0158] For example, if the first value is 0, it indicates that the corresponding couple of first distribution keys is not used to determine the target distribution key. If the first value is not 0 or the first value is greater than or equal to the eighth preset threshold, it indicates that the corresponding couple of first distribution keys is used to determine the target distribution key.

[0159] For example, if a couple of first distribution keys corresponding to target table a and target table b is (kl in table a, kl in table b), two couples of first distribution keys corresponding to target table b and target table c are (k2 in table b, k2 ion table c) and (k3 in table b, k4 in table c), the network cost of (kl in table a, kl in table b) is wl, the network cost of (k2 in table b, k2 in table c) is w2, the network cost of (k3 in table b, k4 in table c) is w3, and wl=w2<w3. To ensure that each target table corresponds to one target distribution key and the sum of each product of the network cost of each couple of first distribution keys corresponding to target tables and the first value of this couple of first distribution keys are maximized, the couple of first distribution keys (kl in table a, kl in table b) and (k3 in table b, k4 in table c) are selected. In other words, the first value corresponding to the couple of first distribution keys (kl in table a, kl in table b) is 1, the first value corresponding to the couple of first distribution keys (k2 in table b, k2 in table c) is 0, and the first value corresponding to the couple of first distribution keys (k3 in table b, k4 in table c) is 1. The target distribution key corresponding to target table a is a single-column distribution key, which is determined based on the first distribution key kl . The target distribution key corresponding to target table b is a multi-column distribution key, which is determined based on the first distribution keys kl and k3. The target distribution key corresponding to target table c is a single-column distribution key, which is determined based on the first distribution key k4. After storing the target table a, the target table b, and the target table c based on the corresponding target distribution keys, when the cloud management platform queries based on each historical join query statement in the set of historical query information, the network costs wl and the network cost w3 are saved, thereby maximizingsaving network costs and improving the efficiency of querying. By analogy, determining the set of target distribution keys based on each couple of first distribution keys that maximizes the sum of each product of the network cost of each couple of first distribution keys corresponding to target tables and the first value corresponding to this couple of first distribution keys, and storing the tables based on the set of target distribution keys save more network costs and improve the efficiency of querying.

[0160] For example, storing target table a based on its corresponding target distribution key includes: hashing the data in the column corresponding to the first distribution key kl in target table a, and storing data with the same hash value in the same storage node. Storing target table b based on its corresponding target distribution key includes: hashing the data in the columns corresponding to the first distribution key kl and k3 in target table b together, and storing data with the same hash value in the same storage node. Storing target table c based on its corresponding target distribution key is similar to storing target table a based on its corresponding target distribution key.

[0161] Optionally, the cloud management platform provides the tenant with the set of target distribution keys, or, provides the tenant with the set of target distribution keys and the first value of each first distribution key corresponding to each target distribution key. The first value of each first distribution key corresponding to the target distribution key is greater than or equal to an eighth preset threshold, and the first value of each first distribution key is the first value of the couple of first distribution keys which includes each first distribution key.

[0162] Optionally, when executing the methods in FIG. 4 or FIG. 5, the cloud management platform provides the tenant with the data obtained in the process of obtaining the set of target distribution keys, and the data obtained in the process of obtaining the set of target distribution keys includes at least one of the following: at least one value corresponding to at least one first attribute of each first distribution key, the network cost of each couple of first distribution keys, the first value of each couple of first distribution keys, the set of target distribution keys, a first storage statement, a sequence of algorithm process query statements and at least one query result corresponding to the sequence of algorithm process query statements. At least one value corresponding to at least one first attribute of each first distribution key includes: estimated value of data skewness and / or query running time. The first storage statement based on querylanguage is used to store the set of target tables based on the set of target distribution keys, and each algorithm process query statement in the sequence of algorithm process query statements is used to query the set of tables and / or the set of historical query information.

[0163] In a possible design, each algorithm process query statement in the sequence of algorithm process query statements is a structured query language (SQL) query statement. The embodiments of the preset application do not limit the language of the query statement.

[0164] In a possible design, a first algorithm process query statement is used to process at least one of the following: at least one table with statistical information, the set of historical query information. Alternatively, the first algorithm process query statement is used to filter based on the value corresponding to at least one first attribute of a distribution key (e.g. estimated value of data skewness and / or query running time). The first algorithm process query statement belongs to the sequence of algorithm process query statements.

[0165] For example, the table with statistical information includes at least one of the following: cardinality of the table, a percentage of an undefined value in the target table, or the number of unique values in the target table. The table with statistical information is imported from database management system (DBMS) technical table.

[0166] For example, the set of historical query information includes at least one of the following: query number of at least one historical query statement, query running time of at least one historical query statement, table being joined in at least one historical join query statement, cardinality of table in each historical join query statement before first join, etc.

[0167] In a possible design, the query result corresponding to the first algorithm process query statement is a new table, which is saved as a temporary result. For example, this new data table is used as inputs in future queries.

[0168] Optionally, the request in step 410 also includes second indication information. The second indication information is used to indicate the path of the target file. The path of the target file is used to access this target file. The target file includes at least one of the following: the set of target distribution keys, the first value of each couple of target distribution keys, at least one value corresponding to at least one first attribute of each first distribution key, the network cost of each couple of target distribution keys, the first storage statement, the sequence of algorithm process query statements and at least one query result corresponding to the sequence ofalgorithm process query statements.

[0169] Optionally, when executing the methods in FIG. 4 or FIG. 5, the cloud management platform receives a second request from the tenant, which is used to obtain first information. The first information includes at least one of the following: the execution progress of the method, estimated waiting time, and the target file mentioned above. The execution progress of this method is the execution progress of the application executing the method in the embodiments of the present application, and the estimated waiting time is the estimated time required to obtain the set of target distribution keys.

[0170] Optionally, the first optimization rule is flexibly adjusted at any time during the execution of the method in the embodiments of the present application. For example, the set of first attributes and / or the set of preset conditions is flexibly adjusted at any time.

[0171] According to the method in FIG. 5, the distribution key with higher estimated value of data skewness is removed from the set of candidate distribution keys, thereby avoiding tenants from storing the set of target tables based on the distribution key with higher estimated value of data skewness, and improving the efficiency of querying. Moreover, according to this method, the query running time corresponding to the set of candidate distribution keys is simulated to avoid obtaining the distribution key with a longer query running time, thereby improving the efficiency of querying after storing tables based on the set of target distribution keys. Meanwhile, according to this method, the set of target distribution keys that satisfies the first constraint condition is obtained from the set of first distribution keys based on the network cost of each couple of first distribution keys, and a single-column distribution key or a multi- column distribution key is determined for each target table, thereby minimizing the network cost of querying after storing tables based on the set of target distribution keys and improving the efficiency of querying data.

[0172] FIG. 6 is a schematic diagram of a device 600 for obtaining a distribution key according to an embodiment of this application. As shown in FIG. 6, the device 600 for obtaining a distribution key includes: a transceiver unit 610 and a processing unit 620. The device 600 for obtaining a distribution key implements the method as shown in FIG. 4-FIG. 5. The device 600 for obtaining a distribution key is applied to cloud management platforms.

[0173] The transceiver unit 610 is configured to receive a request from a tenant. Thetransceiver unit 610 performs the step 410 in FIG. 4.

[0174] The processing unit 620 is configured to: obtain a set of target tables and a set of candidate distribution keys based on the request and a set of historical query information; obtain a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule; and obtain the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys. The processing unit 620 performs the step 420-440 in FIG. 4 and the step 510-530 in FIG. 5.

[0175] Among them, both the transceiver unit 610 and the processing unit 620 are implemented through software or hardware. The processing unit 620 is taken as an example to introduce the implementation of the processing unit 620. Similarly, the implementation of the transceiver unit 610 refers to the implementation of the processing unit 620.

[0176] When the unit is an example of a software functional unit, the processing unit 620 includes code running on a computational instance. Among them, the computational instance includes at least one of physical hosts (computing devices), virtual machines, and containers. Furthermore, the above computational instance can be one or more. For example, the processing unit 620 includes code running on multiple hosts / virtual machines / containers. It should be noted that multiple hosts / virtual machines / containers used to run the code are distributed in the same region or in different regions. Furthermore, multiple hosts / virtual machines / containers used to run the code are distributed within the same availability zone (AZ) or across different AZs, each of which includes a data center or multiple geographically close data centers. Typically, a region includes multiple AZs.

[0177] Similarly, multiple hosts / virtual machines / containers used to run the code are distributed within the same virtual private cloud (VPC) or across multiple VPCs. Among them, usually one VPC is set within a region, and cross regional communication between two VPCs within the same region, as well as VPCs from different regions, requires a communication gateway to be set up within each VPC to achieve interconnection between VPCs.

[0178] When the unit is an example of a hardware functional unit, the processing unit 620 includes at least one computing device, such as a server. Alternatively, the processing unit 620 also is a device implemented using application specific integrated circuits (ASIC) or programmable logic devices (PLD). Among them, the above-mentioned PLD is a complexprogrammable logic device (CPLD), field programmable gate array (FPGA), general array logic (GAL), or any combination thereof.

[0179] The multiple computing devices included in the processing unit 620 are distributed in the same region or in different regions. The multiple computing devices included in the processing unit 620 are distributed within the same AZ or across different AZs. Similarly, the multiple computing devices included in the processing unit 620 are distributed within the same VPC or across multiple VPCs. Among them, the multiple computing devices are any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0180] Therefore, the units of each example described in the embodiments of the present application are implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians may use different methods to achieve the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] It should be noted that the device provided in the above embodiments only provides examples of the division of various functional units when executing the above methods. In practical applications, the above functions can be assigned to different functional units according to needs, that is, the internal structure of the device can be divided into different functional units to complete all or part of the functions described above. For example, the transceiver unit 610 can be used to perform any step in the above method, and the processing unit 620 can be used to perform any step in the above method. The steps responsible for implementing the transceiver unit 610 and the processing unit 620 can be specified as needed. The transceiver unit 610 and the processing unit 620 respectively implement different steps in the above methods to achieve all the functions of the above device.

[0182] In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments mentioned above, which will not be repeated here.

[0183] The method provided in the embodiments of the present application may be executed by a computing device, which may also be referred to as a computer system. Thisincludes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. This hardware layer includes hardware such as processing units, memory, and memory control units, followed by a detailed explanation of the hardware’s functions and structure. This operating system is any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. This application layer includes applications such as browsers, contacts, word processing software, instant messaging software, etc. And, alternatively, the computer system can be a handheld device such as a smartphone, or a terminal device such as a personal computer, which is not specifically limited by the present application, as long as it can be implemented through the methods provided in the embodiments of the present application. The execution subject of the method provided in the embodiments of the present application can be a computing device, or a functional module in the computing device that can call and execute the program.

[0184] FIG. 7 is a schematic block diagram of a computing device according to an embodiment of this application. The computing device 700 can be a server, a computer, or other device with computing power. The computing device 700 shown in FIG. 7 includes at least one processor 710 and a memory 720.

[0185] It should be understood that embodiments of the present application do not limit the number of processors and memory in the computing device 700.

[0186] The processor 710 executes instructions in the memory 720 to enable the computing device 700 to implement the method provided in the embodiments of the present application. Alternatively, the processor 710 executes instructions in the memory 720 to enable the computing device 700 to implement the various functional modules provided in the embodiments of the present application, thereby implementing the methods provided in the embodiments of the present application.

[0187] Optionally, the computing device 700 also includes a communication interface 730. The communication interface 730 uses transceiver modules such as but not limited to network interface cards and transceivers to achieve communication between the computing device 700 and other devices or communication networks.

[0188] Optionally, the computing device 700 also includes a system bus 740, where the processor 710, the memory 720, and the communication interface 730 are respectively connected to the system bus 740. The processor 710 can access the memory 720 through the system bus 740, for example, the processor 710 can read and write data or execute code in the memory 720 through the system bus 740. The system bus 740 is either a peripheral component interconnect express (PCI) bus or an extended industry standard architecture (EISA) bus. The system bus 740 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in FIG. 7, but it does not mean that there is only one bus or one type of bus.

[0189] One possible implementation is that the function of the processor 710 is mainly to interpret instructions (or code) of computer programs and process data in computer software. Among them, the instructions of the computer program and the data in the computer software can be stored in the cache of the memory 720 or the processor 710.

[0190] Optionally, the processor 710 may be an integrated circuit chip with signal processing capabilities. As an example rather than a limitation, the processor 710 is a general- purpose processor, digital signal processor (DSP), ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. Among them, general-purpose processors are microprocessors, etc. For example, the processor 710 is a central processing unit (CPU).

[0191] The memory 720 can provide running space for processes in the computing device 700, for example, storing computer programs (specifically, program code) used to generate processes in the memory 720. After the computer program is run by the processor and generates a process, the processor allocates corresponding storage space for the process in the memory 720. Furthermore, the above storage space further includes text segments, initialization data segments, bit initialization data segments, stack segments, heap segments, and so on. The memory 720 stores data generated during the operation of the process, such as intermediate data or process data, in the storage space corresponding to the above process.

[0192] Optionally, the memory is used to temporarily store operational data in the processor 710 and data exchanged with external memory such as hard drives. As long as the computer is running, the processor 710 will transfer the data that needs to be processed to memory forprocessing, and then transmit the result after the operation is completed.

[0193] As an example rather than a limitation, the memory 720 may be either volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Among them, non-volatile memory is read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), flash, or electrically EPROM (EEPROM). Volatile memory is a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus DRAM (DRDRAM). It should be noted that the memory 720 of the system and method described in this application is intended to include but not limited to these and any other suitable types of memory.

[0194] The structure of the computing device 700 listed above is only for illustrative purposes, and this application is not limited to it. The computing device 700 in the embodiments of this application includes various hardware in computer systems in prior art. For example, the computing device 700 also includes other storage devices besides memory 720, such as disk storage, etc. Technicians in this field should understand that the computing device 700 may also include other devices necessary for normal operation. Meanwhile, according to specific needs, technical personnel in this field should understand that the above-mentioned computing device 700 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the above-mentioned computing device 700 may only include the devices necessary to implement the embodiments of the present application, without necessarily including all the devices shown in FIG. 7.

[0195] The embodiment of this application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server. In some embodiments, the computing device may also be a terminal device such as a desktop computer, laptop, or smartphone.

[0196] As shown in FIG. 8, the computing device cluster includes at least one computing device 700. The memory 720 in one or more computing devices 700 in a computing device cluster may contain the same instructions for executing the above method.

[0197] In some possible implementations, the memory 720 in one or more computing devices 700 within the computing device cluster may also hold partial instructions for executing the above method. In other words, a combination of one or more computing devices 700 can jointly execute the instructions of the above method.

[0198] It should be noted that the memory 720 in different computing devices 700 in the computing device cluster can store different instructions, which are used to perform some of the functions of the above-mentioned devices. That is to say, the instructions stored in the memory 720 of different computing devices 700 can realize the function of one or more modules in the above-mentioned device.

[0199] In some possible implementations, one or more computing devices in a computing device cluster can be connected through a network. Among them, the network can be a wide area network, a local area network, or the like. FIG. 9 illustrates a possible implementation approach. As shown in FIG. 9, two computing devices 700 A and 700B are connected through a network. Specifically, the computing devices 700A and 700B are connected to the network through their communication interfaces.

[0200] It should be understood that the functionality of the computing device 700 A shown in FIG. 9 can also be accomplished by multiple computing devices 700. Similarly, the functionality of computing device 700B can also be accomplished by multiple computing devices 700.

[0201] An embodiment of this application provides a computer program product including instructions, which can run on a computing devices cluster or be stored in any available medium. When it is run by a computing device cluster, the computing device cluster is made to execute the methods provided above, or the computing device cluster is made to implement the functions of the devices provided above.

[0202] An embodiment of this application provides a computer readable storage medium including instructions. The computer readable storage medium is any available medium that computing devices can store, or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (such as floppy disks, hard drives, magnetic tapes), optical media (such as digital video disc (DVD)), or semiconductor media (such as solid-state drives), etc. The computer readable storage medium includesinstructions. When the instructions are run on a computer device cluster, the computer device cluster executes the methods provided above.

[0203] An embodiment of this application provides a chip system, where the chip system includes a memory and a processor, the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that a server on which a chip is disposed performs the methods provided above.

[0204] A person of ordinary skill in the art may be aware that, in combination with the examples described in the embodiments disclosed in this specification, units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on particular applications and design constraints of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the scope of this application.

[0205] It may be clearly understood by a person skilled in the art that, for the purpose of convenient and brief description, for a detailed working process of the foregoing system, apparatus, and unit, refer to a corresponding process in the foregoing method embodiment. Details are not described herein again.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the described apparatus embodiment is merely an example. For example, the unit division is merely logical function division and may be other division in actual implementation. For example, a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.

[0207] The units described as separate parts may be or may not be physically separate, and parts displayed as units may be or may not be physical units, may be located in one position, ormay be distributed on a plurality of network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0208] In addition, functional units in the embodiments of this application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.

[0209] When the functions are implemented in a form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer readable storage medium. Based on such an understanding, the technical solutions in this application essentially, or the part contributing to the prior art, or some of the technical solutions may be implemented in a form of a software product. The computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of this application. The foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk, or an optical disc.The foregoing descriptions are merely specific implementations of this application, but are not intended to limit the protection scope of this application. Any variation or replacement readily figured out by a person skilled in the art within the technical scope disclosed in this application shall fall within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

CLAIMSWhat is claimed is:

1. A method of obtaining a distribution key, wherein the method is applied to a cloud management platform, the cloud management platform manages an infrastructure used to provide cloud services, the infrastructure comprises at least one data center, each data center in the at least one data center comprises storage nodes, and the method comprises: receiving a request from a tenant, wherein the request is used to obtain a set of target distribution keys, and each target distribution key in the set of target distribution keys is used to store a set of tables of the tenant in the storage nodes; obtaining a set of target tables and a set of candidate distribution keys based on the request and a set of historical query information, wherein the set of historical query information comprises at least one historical join query statement, each historical join query statement of the at least one historical join query statement is used to query data in at least two target tables in the set of target tables, and a join key in each historical join query statement is a candidate distribution key for each target table corresponding to each historical join query statement; obtaining a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule, wherein the first optimization rule comprises a set of preset conditions corresponding to a set of first attributes of a distribution key, each preset condition in the set of preset conditions corresponds to a first attribute in the set of first attributes, and each first attribute corresponding to the set of first distribution keys satisfies the preset condition corresponding to the first attribute; and, obtaining the set of target distribution keys from the set of first distribution keys based on a network cost of each couple of first distribution keys, wherein each target distribution key in the set of target distribution keys satisfies a first constraint condition, each target table in the set of target tables corresponds to a target distribution key in the set of target distribution keys, each target distribution key in the set of target distribution keys is a single-column distribution key or a multi-column distribution key, the network cost of each couple of first distribution keys in the set of first distribution keys is determined based on the network cost of at least one historicaljoin query statement corresponding to the couple of first distribution keys, and the couple of first distribution keys is determined based on the at least one historical join query statement.

2. The method according to claim 1, wherein the set of first attributes comprises: estimated value of data skewness and / or query running time, the estimated value of data skewness corresponding to each candidate distribution key is used to estimate unevenness of distribution on data when at least one target table corresponding to each candidate distribution key is stored in the storage nodes based on the candidate distribution key, and the query running time corresponding to the set of candidate distribution keys comprises each time required to query according to each historical join query statement after storage of the set of target tables is simulated based on the set of candidate distribution keys.

3. The method according to claim 2, wherein the obtaining a set of first distribution keys from the set of candidate distribution keys based on a first optimization rule comprises: obtaining the set of first distribution keys from the set of candidate distribution keys based on the estimated value of data skewness and / or query running time corresponding to the set of candidate distribution keys, wherein the estimated value of data skewness of each first distribution key is less than or equal to a first preset threshold, and / or, the query running time corresponding to the set of first distribution keys satisfies a first preset condition in the set of preset conditions.

4. The method according to claim 3, wherein the obtaining the set of first distribution keys from the set of candidate distribution keys based on the estimated value of data skewness and / or query running time corresponding to the set of candidate distribution keys comprises: obtaining a set of second distribution keys from the set of candidate distribution keys based on the estimated value of data skewness of each candidate distribution key, wherein the estimated value of data skewness of each second distribution key is less than or equal to a second preset threshold; and, obtaining the set of first distribution keys from the set of second distribution keys based on the query running time corresponding to the set of second distribution keys, wherein the query running time corresponding to the set of first distribution keys satisfies the first preset condition.

5. The method according to claim 4, wherein the obtaining the set of first distribution keysfrom the set of second distribution keys based on the query running time corresponding to the set of second distribution keys comprises: when the query running time corresponding to the set of second distribution keys does not satisfy the first preset condition, obtaining the set of first distribution keys from the set of second distribution keys, wherein the estimated value of data skewness of each first distribution key is less than or equal to a third preset threshold, and the third preset threshold is less than the second preset threshold; or, when the query running time corresponding to the set of second distribution keys satisfies the first preset condition, determining the set of second distribution keys as the set of first distribution keys.

6. The method according to any one of claims 3-6, wherein the first preset condition comprises at least one of the following: a sum of query running time corresponding to the set of first distribution keys is less than or equal to a fourth preset threshold; or, a sum of query running time corresponding to the set of first distribution keys is less than or equal to a sum of query running time corresponding to a set of original distribution keys; or, a difference between a sum of query running time corresponding to the set of first distribution keys and a sum of query running time corresponding to a set of original distribution keys is less than or equal to a fifth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a sixth preset threshold; or, a first query running time corresponding to the set of first distribution keys is less than or equal to a second query running time corresponding to a set of original distribution keys; or, a difference between a first query running time corresponding to the set of first distribution keys and a second query running time corresponding to a set of original distribution keys is less than or equal to a seventh preset threshold; wherein each original distribution key in the set of original distribution keys is a distribution key used by the tenant to store the set of tables, the first query running time is determined based on the query running time corresponding to the set of first distribution keys, and the second query running time is determined based on the query running time correspondingto a set of original distribution keys.

7. The method according to any one of claims 1-6, wherein the first constraint condition comprises: the set of target distribution keys comprises each target distribution key corresponding to each target table in the set of target tables, and each target distribution key is determined based on at least one first distribution key; the at least one first distribution key corresponding to a target distribution key corresponding to a first target table is determined based on at most one couple of first distribution keys corresponding to the first target table and each second target table in the set of second target tables, the first target table belongs to the set of target tables, each second target table and the first target table are associated target tables, and the associated target tables are tables queried through the same historical join query statement in the set of historical query information; and, each first distribution key corresponding to the target distribution key is the distribution key in the set of first distribution keys that maximizes a value of , wherein Ais the set of target tables, a and b are different target tables in A, W‘bis a network cost of a ith couple of first distribution keys corresponding to a and b, Xa‘bis a first value of the ith couple of first distribution keys corresponding to a and b, W‘bis a non-negative number, 0 < Xa'b< 1 , 1 < i < 7^ , lab is the number of couple of first distribution keys corresponding to a and b, and lab is a positive integer.

8. The method according to claim 7, wherein the first value of each first distribution key corresponding to the target distribution key is greater than or equal to an eighth preset threshold, the first value of each first distribution key is a first value of a couple of first distribution keys which comprises each first distribution key, and the method further comprises: providing the tenant with the set of target distribution keys, or, providing the tenant with the set of target distribution keys and the first value of each first distribution key corresponding to each target distribution key.

9. The method according to any one of claims 1-8, wherein the method further comprises:obtaining a statistic of each target table from a database of the tenant, wherein the statistic of each target table comprises at least one of the following: a size of the target table, a percentage of an undefined value in the target table, or the number of unique values in the target table; and, obtaining the estimated value of data skewness of each candidate distribution key based on the statistic of each target table and the set of candidate distribution keys.

10. The method according to any one of claims 1-9, wherein the method further comprises: providing the tenant with at least one of the following: at least one value corresponding to at least one first attribute of each first distribution key, the network cost of each couple of first distribution keys, a first storage statement, a sequence of algorithm process query statements and at least one query result corresponding to the sequence of algorithm process query statements, wherein the first storage statement based on query language is used to store the set of target tables based on the set of target distribution keys, each algorithm process query statement in the sequence of algorithm process query statements is used to query the set of tables and / or the set of historical query information.

11. A device for obtaining a distribution key, wherein the device is applied to a cloud management platform, the cloud management platform manages an infrastructure used to provide cloud services, the infrastructure comprises at least one data center, each data center in the at least one data center comprises storage nodes, and the device comprises units to perform the method according to any one of claims 1-10.

12. A computing device cluster, comprising at least one computing device, wherein the computing device comprises a processor and a memory coupled with the processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke and run the computer program stored in the memory, so that the computing device executes the method according to any one of claims 1-10.

13. A computer program product comprising instructions, wherein when the computer program product is run on a server, the server is enabled to perform the method according to any one of claims 1-10.

14. A computer readable storage medium storing instructions, wherein when run on a server, the computer readable storage medium enables the server to perform the method according to any one of claims 1-10.

Images