Method and device for determining predicted resource consumption of database commands

Abstract

An embodiment of the present invention relates to a method and apparatus for determining the predicted resource consumption of a database command. The method includes: determining a table called in a database command; executing a command in sampled data of the table, and determining the number of sampled scan rows in which the command runs in the sampled data; determining the sampling index value inclination rate of the sampled data based on the distribution pattern of the sampled data and the number of sampled scan rows; determining the overall index value inclination rate of the overall data of the table based on the sampling index value inclination rate; determining the overall predicted scan row number of the overall data based on the overall index value inclination rate; the overall predicted scan row number is used to determine the predicted resource consumption of the command running in the overall data. The overall predicted scan row number determined based on the overall index value inclination rate takes into account the changes in the distribution balance of the overall data relative to the sampled data, so that the overall predicted scan row number is more accurate, which can improve the accuracy of determining the predicted resource consumption.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer vision technology, and in particular to a method, apparatus, computing device, and computer-readable storage medium for determining predicted resource consumption of database commands. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] When executing commands, the database may be overloaded due to excessive resource consumption by certain commands, causing command execution to become slow. In severe cases, the server where the database is located may crash, making the database unable to provide external services.

[0004] Being able to predict the resource consumption of commands can prevent database overloads or quickly troubleshoot problems when they occur. For example, the predicted resource consumption of any command can be calculated before execution. If the predicted resource consumption is too high, the command will not be executed. Alternatively, when a database overload occurs, the predicted resource consumption of each command can be used to accurately locate the command that caused the database failure, allowing the command to be processed accordingly to restore normal operation.

[0005] In summary, a method for determining the predicted resource consumption of a command in a database is provided, thereby achieving high availability of the database. Summary of the Invention

[0006] An embodiment of the present invention provides a method for determining predicted resource consumption of a database command, so as to achieve high availability of the database.

[0007] In a first aspect, an embodiment of the present invention provides a method for determining predicted resource consumption of a database command, comprising:

[0008] Determine the table called in the database command;

[0009] Executing the command in the sampled data of the table to determine the number of sampled scan rows in which the command is executed in the sampled data; determining a sampling index value inclination rate of the sampled data based on a distribution pattern of the sampled data and the number of sampled scan rows; the index value inclination rate is used to represent a degree of distribution balance of the data with respect to the index corresponding to the command;

[0010] Determine the overall index value inclination rate of the overall data of the table according to the sampling index value inclination rate;

[0011] The overall predicted number of scan lines of the overall data is determined according to the overall index value slope; the overall predicted number of scan lines is used to determine the predicted resource consumption of the command running in the overall data.

[0012] Since the distribution balance degree of the overall data and the sampled data for the index corresponding to the command is different, there is currently a lack of a way to explore the changes in the distribution balance degree of the overall data and the sampled data respectively. The embodiment of the present application first executes the command in the sampled data of the table corresponding to the command, so that the accurate number of sampled scan rows can be obtained, and the distribution balance degree of the sampled data for the index corresponding to the command is quantified by the number of sampled scan rows and the distribution law of the sampled data, that is, the sampling index value inclination rate is obtained. Then, the overall index value inclination rate of the overall data can be determined, that is, the distribution balance degree of the overall data for the index corresponding to the command. In this way, the overall predicted number of scan rows determined according to the overall index value inclination rate will take into account the changes in the distribution balance degree of the overall data relative to the sampled data, so that the overall predicted number of scan rows is more accurate, which can improve the accuracy of determining the predicted resource consumption.

[0013] In some embodiments, the sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data;

[0014] Determining the overall index value inclination rate of the overall data of the table according to the sampling index value inclination rate includes:

[0015] For any of the first sub-sampled data, determining a first index value inclination rate of the first sub-sampled data according to the number of first sub-sampled scan lines corresponding to the first sub-sampled data and a distribution rule of the first sub-sampled data;

[0016] For any second sub-sampled data, determining a second index value inclination rate of the second sub-sampled data according to the number of second sub-sampled scan lines corresponding to the second sub-sampled data and a distribution rule of the second sub-sampled data;

[0017] Determine a change rule of the index value slope from the second sub-sampled data to the sampled data according to the second index value slope, the first index value slope, and the sampled index value slope;

[0018] The overall index value inclination is determined according to the second index value inclination and a change rule of the index value inclination.

[0019] The sampled data is divided into w first sub-sampled data, each of which contains w second sub-sampled data. For the first sub-sampled data, the first index value slope of the first sub-sampled data can be determined based on the number of first sub-sampled scan lines corresponding to the first sub-sampled data and the distribution pattern of the first sub-sampled data. For the second sub-sampled data, the second index value slope of the second sub-sampled data can be determined based on the number of second sub-sampled scan lines corresponding to the second sub-sampled data and the distribution pattern of the second sub-sampled data. The variation pattern of the index value slope can then be summarized based on the second index value slope corresponding to the second sub-sampled data, the first index value slope corresponding to the first sub-sampled data, and the sampling index value slope corresponding to the sampled data. The overall index value slope can then be determined based on the second index value slope and the variation pattern of the index value slope, thereby improving the accuracy of the overall index value slope, and thereby improving the accuracy of the overall predicted number of scan lines for the overall data.

[0020] In some embodiments, determining the overall predicted number of scan lines of the overall data according to the overall index value slope includes:

[0021] Determine the overall discrimination score of the overall data for the index based on the sampling discrimination score of the sampled data for the index; the discrimination score is used to represent the number of different values ​​of the index in the data;

[0022] Determining the distribution pattern of the overall data according to the overall distinction number and the data volume of the overall data;

[0023] The overall predicted number of scan lines is determined according to the distribution pattern of the overall data and the overall index value inclination.

[0024] By introducing the concept of a discrimination score, the overall discrimination score of the overall data for the index determined based on the sampling discrimination score of the sampled data for the index is relatively accurate. Therefore, determining the distribution pattern of the overall data based on the relatively accurate overall discrimination score and the overall data volume can improve the accuracy of the distribution pattern of the determined overall data. This in turn improves the accuracy of the determination of the second scan line number.

[0025] In some embodiments, the sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data;

[0026] Determining the overall discrimination score of the overall data for the index according to the sampling discrimination score of the sampled data for the index includes:

[0027] determining a first non-overlapping rate of the sampled data for the index based on the w first sub-sampled data and the sampled data; the first non-overlapping rate is used to represent a proportion of the number of distinctions retained in the process of merging the w first sub-sampled data into the sampled data;

[0028] determining a second non-overlapping ratio of the first sub-sampled data with respect to the index based on the w second sub-sampled data and the first sub-sampled data corresponding to the w second sub-sampled data; the second non-overlapping ratio is used to represent a proportion of the number of distinctions retained in the process of merging the w second sub-sampled data into the first sub-sampled data;

[0029] Determine a variation rule of the non-overlapping rate from the first sub-sampled data to the sampled data according to the first non-overlapping rate and the second non-overlapping rate;

[0030] The overall discrimination number is determined according to the second non-overlapping rate, the variation rule of the non-overlapping rate, the data volume of the overall data, the sampling discrimination number and the data volume of the sampling data.

[0031] The sampling data is divided to obtain w first sub-sampling data, where each first sub-sampling data contains w second sub-sampling data. Then, the first non-overlapping rate of the first sub-sampling data and the second non-overlapping rate of the sampling data can be determined, and then the changing rule of the non-overlapping rate from the first sub-sampling data to the sampling data can be summarized. In this way, the overall discrimination score can be determined more accurately based on the summarized changing rule of the non-overlapping rate.

[0032] In some embodiments, the distribution pattern of the sampled data is the average amount of sampled data corresponding to each sampling zone score in the sampled data.

[0033] In some embodiments, determining a first non-overlapping rate of the sampled data for the index based on the w first sub-sampled data and the sampled data includes:

[0034] The ratio of the sampling discrimination number of the sampled data with respect to the index to the sum of the first discrimination numbers of the w first sub-sampled data with respect to the index is used as the first non-overlapping rate.

[0035] In this way, the proportion of the number of distinctions retained in the process of merging the w first sub-sampled data into the sampled data can be more accurately represented.

[0036] In some embodiments, the i-th second sub-sampled data among the w2 second sub-sampled data is determined by:

[0037] The entire data is sharded in the order of primary keys to obtain s shards; the amount of data in each shard is the same;

[0038] Divide the s shards into z groups according to the rule that each e shards constitutes a group;

[0039] For any of the z groups, select w 2 shards;

[0040] From the w 2 f data are selected from the i-th slice of the w2 slices as a part of the i-th second sub-sampled data of the w2 second sub-sampled data.

[0041] Compared to random sampling, this sampling method ensures that the sampled data is evenly distributed. Based on this evenly distributed sampled data, a more accurate sampling index value slope can be determined, and thus a more accurate overall index value slope can be determined. This improves the accuracy of the final overall predicted number of scan rows.

[0042] In some embodiments, the w first sub-sampled data are determined by:

[0043] The w 2 Any w second sub-sampled data among the w second sub-sampled data are merged to obtain the w first sub-sampled data; wherein the second sub-sampled data included in each first sub-sampled data is not repeated.

[0044] In this way, by establishing a deterministic relationship between the second sub-sampling data and the first sub-sampling data, the changing pattern of the index value slope rate or the changing pattern of the non-overlapping rate can be determined more simply and accurately.

[0045] In some embodiments, the sampling data is determined by:

[0046] The w first sub-sampled data are combined to obtain the sampled data.

[0047] In this way, by establishing a deterministic relationship between the first sub-sampled data and the sampled data, the changing pattern of the index value slope rate or the changing pattern of the non-overlapping rate can be determined more simply and accurately.

[0048] In a second aspect, an embodiment of the present invention further provides a device for determining predicted resource consumption of a database command, comprising:

[0049] Identify units for:

[0050] Determine the table called in the database command;

[0051] Executing the command in the sampled data of the table to determine the number of sampled scan rows in which the command is executed in the sampled data; determining a sampling index value inclination rate of the sampled data based on a distribution pattern of the sampled data and the number of sampled scan rows; the index value inclination rate is used to represent a degree of distribution balance of the data with respect to the index corresponding to the command;

[0052] Determine the overall index value inclination rate of the overall data of the table according to the sampling index value inclination rate;

[0053] The overall predicted number of scan lines of the overall data is determined according to the overall index value slope; the overall predicted number of scan lines is used to determine the predicted resource consumption of the command running in the overall data.

[0054] In some embodiments, the sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data;

[0055] The determining unit is specifically configured to:

[0056] For any of the first sub-sampled data, determining a first index value inclination rate of the first sub-sampled data according to the number of first sub-sampled scan lines corresponding to the first sub-sampled data and a distribution rule of the first sub-sampled data;

[0057] For any second sub-sampled data, determining a second index value inclination rate of the second sub-sampled data according to the number of second sub-sampled scan lines corresponding to the second sub-sampled data and a distribution rule of the second sub-sampled data;

[0058] Determine a change rule of the index value slope from the second sub-sampled data to the sampled data according to the second index value slope, the first index value slope, and the sampled index value slope;

[0059] The overall index value inclination is determined according to the second index value inclination and a change rule of the index value inclination.

[0060] In some embodiments, the determining unit is specifically configured to:

[0061] Determine the overall discrimination score of the overall data for the index based on the sampling discrimination score of the sampled data for the index; the discrimination score is used to represent the number of different values ​​of the index in the data;

[0062] Determining the distribution pattern of the overall data according to the overall distinction number and the data volume of the overall data;

[0063] The overall predicted number of scan lines is determined according to the distribution pattern of the overall data and the overall index value inclination.

[0064] In some embodiments, the sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data;

[0065] The determining unit is specifically configured to:

[0066] determining a first non-overlapping rate of the sampled data for the index based on the w first sub-sampled data and the sampled data; the first non-overlapping rate is used to represent a proportion of the number of distinctions retained in the process of merging the w first sub-sampled data into the sampled data;

[0067] determining a second non-overlapping ratio of the first sub-sampled data with respect to the index based on the w second sub-sampled data and the first sub-sampled data corresponding to the w second sub-sampled data; the second non-overlapping ratio is used to represent a proportion of the number of distinctions retained in the process of merging the w second sub-sampled data into the first sub-sampled data;

[0068] Determine a variation rule of the non-overlapping rate from the first sub-sampled data to the sampled data according to the first non-overlapping rate and the second non-overlapping rate;

[0069] The overall discrimination number is determined according to the second non-overlapping rate, the variation rule of the non-overlapping rate, the data volume of the overall data, the sampling discrimination number and the data volume of the sampling data.

[0070] In some embodiments, the distribution pattern of the sampled data is the average amount of sampled data corresponding to each sampling zone score in the sampled data.

[0071] In some embodiments, the determining unit is specifically configured to:

[0072] The ratio of the sampling discrimination number of the sampled data with respect to the index to the sum of the first discrimination numbers of the w first sub-sampled data with respect to the index is used as the first non-overlapping rate.

[0073] In some embodiments, the determining unit is specifically configured to:

[0074] The entire data is sharded in the order of primary keys to obtain s shards; the amount of data in each shard is the same;

[0075] Divide the s shards into z groups according to the rule that each e shards constitutes a group;

[0076] For any of the z groups, select w 2 shards;

[0077] From the w 2Select f data from the i-th shard in the shards as the w 2 A part of the i-th second sub-sampled data of the second sub-sampled data.

[0078] In some embodiments, the determining unit is specifically configured to:

[0079] The w 2 Any w second sub-sampled data among the w second sub-sampled data are merged to obtain the w first sub-sampled data; wherein the second sub-sampled data included in each first sub-sampled data is not repeated.

[0080] In some embodiments, the determining unit is specifically configured to:

[0081] The w first sub-sampled data are combined to obtain the sampled data.

[0082] In a third aspect, an embodiment of the present invention further provides a computing device, including:

[0083] memory for storing computer programs;

[0084] The processor is configured to call the computer program stored in the memory and execute the method for determining the predicted resource consumption of the database command listed in any of the above-mentioned ways according to the obtained program.

[0085] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute any of the above-mentioned methods for determining the predicted resource consumption of a database command. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0087] Figure 1 A schematic diagram of a system architecture for determining predicted resource consumption of a command provided by an embodiment of the present invention;

[0088] Figure 2 A schematic diagram of a system architecture for determining predicted resource consumption of a command provided by an embodiment of the present invention;

[0089] Figure 3 A schematic diagram of a method for determining predicted resource consumption of a database command provided by an embodiment of the present invention;

[0090] Figure 4 A schematic diagram of the association relationship between various sample data provided by an embodiment of the present invention;

[0091] Figure 5 A schematic diagram of determining the overall discrimination score of overall data with respect to an index by using the sampling discrimination score of sampled data with respect to an index provided by an embodiment of the present invention;

[0092] Figure 6 A schematic diagram of a first non-overlapping rate of the sampled data with respect to the index and a second non-overlapping rate of the first sub-sampled data with respect to the index provided in an embodiment of the present invention;

[0093] Figure 7 A schematic diagram of a database command provided by an embodiment of the present invention;

[0094] Figure 8 A schematic diagram of a method for determining the overall index value inclination rate of overall data provided by an embodiment of the present invention;

[0095] Figure 9 A schematic diagram of an execution plan provided by an embodiment of the present invention;

[0096] Figure 10 A schematic diagram of the index value inclination rates corresponding to the second sub-sampled data, the first sub-sampled data, and the sampled data provided in an embodiment of the present invention;

[0097] Figure 11 A schematic structural diagram of a device for determining predicted resource consumption of a database command provided by an embodiment of the present invention;

[0098] Figure 12 A schematic structural diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0099] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0100] All other embodiments derived by persons of ordinary skill in the art based on the exemplary embodiments described herein without inventive effort are within the scope of protection of the claims appended hereto. Furthermore, although the disclosure herein is presented based on one or more exemplary embodiments, it should be understood that each aspect of the disclosure may constitute a complete embodiment on its own.

[0101] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0102] In the specification and claims of this application and the drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or precedence, unless otherwise indicated. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances, for example, the embodiments of this application can be implemented in an order other than those shown or described in the drawings.

[0103] In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0104] Figure 1 The system architecture diagram of determining the predicted resource consumption of a command provided by an embodiment of the present application is shown, including a database 100. The database 100 stores multiple tables, such as Table A, Table B, Table C, and Table D shown in the figure.

[0105] The database 100 samples data from each table to obtain sampled data for each table, for example, randomly extracting 20 pages of sampled data from each table. Based on the distribution pattern of the sampled data from each table, the database 100 predicts the distribution pattern of the overall data in the table to obtain statistical information for the table. The statistical information is the simulated overall data of the table. For any received command, the database 100 determines at least one table called in the command. For example, if the command queries Table A, the called table is Table A. The database 100 executes the command based on the statistical information of the at least one table called by the command to obtain an execution plan for the command. The execution plan includes a predicted number of scan rows required to execute the command. The predicted number of scan rows is the number of scan rows that may be required to execute the command in the overall data. Based on the predicted number of scan rows, the database 100 determines the predicted resource consumption required to execute the command in the overall data. A greater number of predicted scan rows indicates a higher predicted resource consumption.

[0106] However, the above method has the following problems: 1. The random sampling method is not scientific and reasonable. If the sampled data is too small, the sampled data will not be discrete enough. When the overall data in the table is not sufficiently distinguishable for a certain index, or when there is data skew, the sampled data obtained by random sampling cannot accurately reflect the distribution pattern of the overall data in the table. The statistical information of the table is inaccurate, and the predicted resource consumption is inaccurate. If the sampled data is too large, the analysis pressure of the statistical information obtained based on the sampled data is high, which occupies more computing resources, resulting in less computing resources available for the database to provide external services, and the database may malfunction. 2. The method of first determining the statistical information based on the sampled data and then executing the command in the statistical information to determine the predicted number of scan rows is relatively simple and does not take into account more complex information. Therefore, the predicted number of scan rows determined is inaccurate, and the predicted resource consumption is also inaccurate.

[0107] Figure 2 A system architecture diagram for determining predicted resource consumption of another command provided in an embodiment of the present application is shown, including a database 100 and a database 200 .

[0108] Database 100 stores multiple tables, such as Table A, Table B, Table C, and Table D shown in the figure. Data is sampled from each table in database 100 to obtain sampled data, which is then stored in a dedicated database instance, namely, database 200. By storing the sampled data in database 200, it can be analyzed within database 200 to obtain predicted resource consumption. This reduces the resource usage of database 100 during the analysis process, minimizing the impact on database 100 and enabling it to provide normal external services.

[0109] For any command received, at least one table called in the command is determined. For example, if the command queries Table A, the table called is Table A. The command is executed in the sampled data of at least one table called by the command to obtain an execution plan for the command. The execution plan includes a sampled scan line number for executing the command, and the sampled scan line number is an accurate scan line number. Based on the accurate scan line number and the accurate distribution law of the sampled data, an accurate sampled index value inclination rate of the sampled data can be obtained. The index value inclination rate is used to characterize the degree of distribution balance of the data for the index corresponding to the command. Based on the sampled index value inclination rate, the overall index value inclination rate of the overall data of the table can be estimated. Based on the estimated overall index value inclination rate, the overall predicted scan line number of the overall data is determined. Since the index value inclination rate is taken into account, the overall predicted scan line number is more accurate. Therefore, the predicted resource consumption of the command running in the overall data is determined by the more accurate overall predicted scan line number, which can improve the accuracy of determining the predicted resource consumption of the command running in the overall data.

[0110] Optionally, if the amount of sampled data is small, the above-mentioned execution steps in database 200 can also be completed in database 100.

[0111] As can be seen, the order of determining the predicted resource consumption differs between the two embodiments. In the second embodiment, the command is first executed on the sampled data to obtain an accurate number of sampled scan lines. The index value slope is introduced to consider the distribution balance of the sampled data to the index and the distribution balance of the overall data to the index. This allows for a more accurate overall predicted number of scan lines, and thus a more accurate predicted resource consumption.

[0112] Database 100 and database 200 can be deployed in the same server or in different servers. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0113] The following describes a method for determining the predicted resource consumption of a database command provided by an embodiment of the present application. Figure 3 shown.

[0114] Step 301, determining the table called in the database command;

[0115] Step 302: Execute the command in the sampled data of the table to determine the number of sampled scan rows in which the command is executed in the sampled data; determine a sampling index value inclination rate of the sampled data based on the distribution pattern of the sampled data and the number of sampled scan rows; the index value inclination rate is used to indicate the degree of distribution balance of the data with respect to the index corresponding to the command;

[0116] Step 303, determining the overall index value inclination rate of the overall data in the table according to the sampling index value inclination rate;

[0117] Step 304 : determining the overall predicted number of scan lines of the overall data according to the overall index value inclination; the overall predicted number of scan lines is used to determine the predicted resource consumption of the command running in the overall data.

[0118] In step 301, the table called in the database command is determined.

[0119] The command is stored in the database 100 according to Figure 1Execute in the manner shown. Although the predicted number of scanned rows is inaccurate, the table invoked by the command and the index used can be determined. For example, the table invoked in this database command is Table A, and the index used is the "job" field.

[0120] In step 302, to minimize the impact of sample data extraction on database 100, sample data is extracted from a backup database within database 100. The table invoked by the command was determined in step 1, so the overall data is the data in the table invoked by the command. In this example, the overall data can be all the data in Table A.

[0121] The specific method of extracting sample data is as follows: the whole data is sharded in the order of primary key to obtain s shards; the amount of data in each shard is the same; the s shards are divided into z groups according to the rule that each e shard is a group; for any group in the z groups, w 2 fragments; from the w 2 Select f data from the i-th shard among the shards as w 2 The second sub-sample data is a portion of the i-th second sub-sample data. Here, w can be any natural number such as 2, 3, etc. For example, if w is equal to 2, the number of the second sub-sample data is 2; if w is equal to 3, the number of the second sub-sample data is 9.

[0122] For example, the total data is 17 million. Since the table is sorted by primary key, and a data can be uniquely identified by the primary key, it is divided according to the primary key order. Every 100,000 data is a shard, so there are 170 shards in total. Shard data extraction requires determining two values: t shards are taken from s shards for extraction, and the size f of the data extracted from each shard. The number of shards t is required to be w 2 Multiples of s / e facilitate data distribution. t is determined using the following formula: t = MOD(s / e) * 4. e is the number of shards in each group. Starting from 4, e is incremented by 1 until the determined t is less than 100 and closest to 100. The value 100 is an empirical value and can be set arbitrarily by those skilled in the art. For example, in this example, when e = 7, the determined t is less than 100 and closest to 100, t = 96.

[0123] f is determined by the following formula: f = (s*100000) / (2 i *(t / 4)). i is the number of merges from the second subsampled data to the sampled data. Let i start at 1 and increase until f is closest to 500, while being greater than 500. The value 500 here is an empirical value and can be set arbitrarily by those skilled in the art. In this example, when i = 9, f satisfies the condition. That is, f = 170 * 100,000 / (128 * 4 / (96 / 4)) = 1383.

[0124] In summary, the 170 shards were divided into 24 groups of 7 shards each. Four shards were selected from each of the 7 shards in each group, for a total of 96 shards. 1,383 data points were selected from each shard. The selected data was placed into the second sub-sampled data in the following manner: for a group with 4 shards, 1,383 data points were selected from shard 1 and placed into the first second sub-sampled data point, 1,383 data points were selected from shard 2 and placed into the second second sub-sampled data point, 1,383 data points were selected from shard 3 and placed into the third second sub-sampled data point, and 1,383 data points were selected from shard 4 and placed into the fourth second sub-sampled data point. The above steps were repeated for the next group. Thus, each second sub-sampled data point contains data from one shard from each of the 24 groups. This ensures that the data in each second sub-sampled data point is widely dispersed throughout the overall data, ensuring the dispersion of the second sub-sampled data point and better reflecting the distribution pattern of the overall data.

[0125] Figure 4 The diagram shows a correlation relationship between the sampled data provided by the embodiment of the present application. The diagram shows a binary tree diagram. If the second sub-sampled data is 9, the diagram should be a ternary tree diagram. In the above, the data extracted from each slice is placed in Figure 4 The four tables of the third layer shown form four second sub-sample data.

[0126] The w first sub-sample data are determined by the following method, including: 2 Merging any w second sub-sampled data from the w second sub-sampled data to obtain the w first sub-sampled data; wherein the second sub-sampled data included in each first sub-sampled data is non-repeated. Determining the sampled data by the following method includes: merging the w first sub-sampled data to obtain the sampled data.

[0127] In this way, by establishing a deterministic relationship between the second sub-sampling data and the first sub-sampling data, and a deterministic relationship between the first sub-sampling data and the sampling data, the change law of the index value inclination rate or the change law of the non-overlapping rate can be determined more simply and accurately.

[0128] For example, Figure 4 In the example, the second sub-sampled data L2_1 and the second sub-sampled data L2_3 are merged to obtain the first sub-sampled data L1_1; the second sub-sampled data L2_2 and the second sub-sampled data L2_4 are merged to obtain the first sub-sampled data L1_2; the first sub-sampled data L1_1 and the first sub-sampled data L1_2 are merged to obtain the sampling data L0_1.

[0129] Table 1 shows a possible second sub-sample data L2_1. This table contains Index 1 and Index 2, where Index 1 contains two fields: job and flow, and Index 2 contains three fields: status, host, and part. Table 2 shows a possible second sub-sample data L2_3, which has the same table structure as L2_1. Table 2 shows a schematic diagram of the first sub-sample data L1_1 obtained by merging the second sub-sample data L2_1 and the second sub-sample data L2_3.

[0130] Table 1

[0131]

[0132] Table 2

[0133]

[0134] Table 3

[0135]

[0136] The distinct value is used to represent the number of different values ​​in the index in the data. It can be seen that the distinct value of the primary key in Table 1 is 30, because each data point has a different primary key, and there are 30 data points in total; the distinct value of the job field is 4; and the distinct value of the job+flow field is 5. The distinct value of the primary key in Table 2 is 30, because each data point has a different primary key, and there are 30 data points in total; the distinct value of the job field is 3; and the distinct value of the job+flow field is 6. The distinct value of the primary key in Table 2 is 60, because each data point has a different primary key, and there are 60 data points in total; the distinct value of the job field is 6; and the distinct value of the job+flow field is 9.

[0137] It can be found that when two tables are merged into one table, the distinction scores are not a simple addition relationship.

[0138] Table 4 shows the Figure 4 In the schematic structure, the change in the number of distinctions after the tables are merged.

[0139]

[0140]

[0141] Next, by analyzing the changes in the above discrimination scores, the overall discrimination score of the overall data for a certain index is obtained. Figure 5 A schematic diagram is shown for determining the overall discrimination score of the overall data for the index by the sampling discrimination score of the sampled data for the index. The steps include:

[0142] Step 501: Determine a first non-overlapping ratio of the sampling data for the index based on the w first sub-sampled data and the sampling data; the first non-overlapping ratio is used to represent the proportion of the number of distinctions retained in the process of merging the w first sub-sampled data into the sampling data.

[0143] The ratio of the sampling discrimination number of the sampled data with respect to the index to the sum of the first discrimination numbers of the w first sub-sampled data with respect to the index is used as the first non-overlapping rate.

[0144] For example, the discrimination scores of the first sub-sampled data L1_1 and L1_2 for the job field in index 1 are 15883 and 16154 respectively; the discrimination score of the sampled data L0 for the job field in index 1 is 31973; therefore, the first non-overlapping rate = 31973 / (15883+16154) = 0.998.

[0145] That is, in the process of merging the first sub-sampled data into the sampled data, the proportion of the discrimination number retained for the job field in index 1 is 0.998.

[0146] Step 502: Determine a second non-overlapping ratio of the first sub-sampling data for the index based on the w second sub-sampling data and the first sub-sampling data corresponding to the w second sub-sampling data; the second non-overlapping ratio is used to represent the proportion of the number of distinctions retained in the process of merging the w second sub-sampling data into the first sub-sampling data.

[0147] The calculation method is the same as step 501. For example, the discrimination scores for the job field in index 1 of the second sub-sample data L2_1 and L2_3 are 7991 and 7913, respectively; the discrimination score for the job field in index 1 of the first sub-sample data L1_1 is 15883; therefore, the second non-overlapping rate of the first sub-sample data L1_1 = 15883 / (7991+7913) = 0.9987. The discrimination scores for the job field in index 1 of the second sub-sample data L2_2 and L2_4 are 7604 and 8559, respectively; the discrimination score for the job field in index 1 of the first sub-sample data L1_2 is 16154; therefore, the second non-overlapping rate of the first sub-sample data L1_2 = 16154 / (7604+8559) = 0.9994. The average value of the first sub-sample data L1_2 and the second non-overlapping rate of the first sub-sample data L1_2 is (0.9987+0.9994) / 2=0.99905.

[0148] Step 503: Determine a variation pattern of the non-overlapping rate from the first sub-sampled data to the sampled data according to the first non-overlapping rate and the second non-overlapping rate.

[0149] Figure 6 The first non-overlapping rate of the sampled data for the index and the second non-overlapping rate of the first sub-sampled data for the index are shown. With the number of data doublings as the horizontal axis and the non-overlapping rate as the vertical axis, the slope of the non-overlapping rate curve is calculated to be (0.998-0.99905) / (2-1)=-0.00105.

[0150] The slope can represent the variation pattern of the non-overlapping rate from the merging of the first sub-sampled data to the sampled data.

[0151] Step 504: determine the overall discrimination number according to the second non-overlapping rate, the variation rule of the non-overlapping rate, the data volume of the overall data, the sampling discrimination number, and the data volume of the sampling data.

[0152] Figure 4 The diagram shows a three-layer tree structure composed of the second sub-sampled data, the first sub-sampled data and the sampled data. It can be imagined that if the rest of the overall data is sampled multiple times, multiple sub-sampled data can be obtained. Figure 4 The same tree structure diagram, the root nodes of these tree structures are continuously merged upwards to obtain the overall data. The structures of these trees are the same, and the second sampling data that constitute the tree are evenly distributed in the overall data. Therefore, we believe that the data distribution rules of these trees are the same as Figure 4 The distribution law shown is the same. Therefore, there is no need to sample the rest of the overall data multiple times to form multiple trees, but to directly use Figure 4 The tree structure shown, which is composed of the second sub-sampled data, the first sub-sampled data and the sampled data, is used to analyze the data distribution law.

[0153] Given that the total data volume is m and the number of distinctions in L1_1 is n, then merging from L1_1 continuously until the total data is obtained requires i mergings. The average of the discrimination scores of L1_1 and L1_2 for a certain index is recorded as x, the non-overlapping rate corresponding to L1_1 is recorded as k, and the slope is recorded as s (which is the variation law of the non-overlapping rate mentioned above). Then the overall discrimination score y of the overall data for a certain index is calculated by the following formula: y1=x*2 i *(ks*1)(ks*2)…(ks*i)*m / (n*2 i ).

[0154] For example, in the above example, the total data volume m = 17 million, the discrimination score of L1_1 n = 65772, x = (15883 + 16154) / 2 = 16018, and if the data layer where L1_1 is located is continuously merged upward until the total data volume is reached, it will need to go through 8 merges, that is, i = 8, k = 0.99905, s = 0.00105, and the result is that the discrimination score of the job field in index 1 of the total data is y1 = 3.96 million. After actual verification, that is, after running this command on the total data, the actual discrimination score is 3.47 million, and using Figure 1 The method shown in the figure obtains a discrimination score of 2.37 million for the job field in index 1. It can be seen that the discrimination score for a certain index obtained by the method provided in the embodiment of the present application is closer to the true value and more accurate.

[0155] Through the above calculation, the overall number of distinctions for the job field in index 1 is 3.96 million.

[0156] Using the same method, we can get the overall discrimination score of the entire data for index 2 as 154. After actual testing, that is, after running the command on the entire data, the actual discrimination score is 114, while using Figure 1 The discrimination score for index 2 obtained by the illustrated method is 16245. It can be seen that the discrimination score for a certain index obtained by the method provided in the embodiment of the present application is closer to the true value and more accurate.

[0157] The sampling data is divided to obtain w first sub-sampling data, where each first sub-sampling data contains w second sub-sampling data. Then, the first non-overlapping rate of the first sub-sampling data and the second non-overlapping rate of the sampling data can be determined, and then the changing rule of the non-overlapping rate from the first sub-sampling data to the sampling data can be summarized. In this way, the overall discrimination score can be determined more accurately based on the summarized changing rule of the non-overlapping rate.

[0158] In step 303, the overall index value inclination of the overall data in the table is determined according to the sampling index value inclination.

[0159] Because the sampled data is small, the dedicated instance - database 200 can run the command in the sampled data to obtain the number of sampled scan lines of the command in the sampled data; run the command in each first sub-sampled data respectively to obtain the number of first sub-sampled scan lines of the command in each first sub-sampled data; run the command in each second sub-sampled data to obtain the number of second sub-sampled scan lines of the command in each second sub-sampled data.

[0160] Figure 7 This is a schematic diagram of a database command provided in an embodiment of the present application. The command is a SQL command.

[0161] Figure 8 A method for determining the overall index value inclination rate of overall data provided in an embodiment of the present application includes the following steps:

[0162] Step 801: execute this SQL command in the sampled data to obtain the number of sampled scan rows when the command is executed in the sampled data, and determine the slope of the sampled index value of the sampled data according to the distribution law of the sampled data and the number of sampled scan rows.

[0163] For example, when executing this command on the sample data L0, the execution plan obtained is as follows: Figure 9 As shown, Figure 9 ] shows that the number of sample scan lines for executing the command on the sample data L0 is 775. The corresponding execution cost, that is, the resource consumption is 710.21.

[0164] The distribution law of the sampled data is defined as the amount of sampled data corresponding to each sampling partition number in the sampled data. That is, distribution law = total amount of data / partition number.

[0165] The index value slope is defined to represent the distribution balance of data for the index corresponding to the command. Index value slope y2 = number of scanned rows / distribution pattern = number of scanned rows / (total amount of data / number of partitions).

[0166] In the sampled data, the total amount of data is 131544; the number of distinctions for index 2 is 79, and the number of scanned rows is 775, so the obtained sampling index value inclination rate is 775 / (131544 / 79)=0.4564.

[0167] Step 802: Execute this SQL command in any first sub-sampled data to obtain the number of first sub-sampled scan rows when the command is executed in the first sub-sampled data; determine the inclination rate of the first index value of the first sub-sampled data based on the number of first sub-sampled scan rows and the distribution pattern of the first sub-sampled data.

[0168] The calculation method is the same as step 801. For example, when this command is executed in L1_1, the number of first sub-sample scan lines is 390, and the distribution pattern of the first sub-sample data is 65772 / 78. Therefore, the slope of the first index value corresponding to L1_1 is 390 / (65772 / 78) = 0.4625. When this command is executed in L1_2, the number of first sub-sample scan lines is 385, and the distribution pattern of the first sub-sample data is 65772 / 67. Therefore, the slope of the first index value corresponding to L1_2 is 385 / (65772 / 67) = 0.3922.

[0169] The average value of the slope of the first index value of the first sub-sampled data is 0.42735.

[0170] Step 803: Execute this SQL command in any second sub-sampled data to obtain the number of second sub-sampled scan rows when the command is executed in the second sub-sampled data; and determine the inclination rate of the second index value of the second sub-sampled data based on the number of second sub-sampled scan rows and the distribution pattern of the second sub-sampled data.

[0171] The calculation method is the same as step 801. The average value of the slope of the second index value of the second sub-sample data is 0.389.

[0172] Step 804: Determine a variation rule of the index value slope from the second sub-sampled data to the sampled data according to the second index value slope, the first index value slope, and the sampled index value slope.

[0173] With the index value slope as the vertical axis and the number of table merges as the horizontal axis, we obtain the first slope k0 from the second index value slope to the first index value slope, and the second slope k1 from the first index value slope to the sampled index value slope. The rate of change of the slope is determined based on the first and second slopes. Assuming the rate of change of the slope is constant, the rate of change of the slope represents the changing pattern of the index value unloading rate.

[0174] We believe that the distribution balance corresponding to the second subsample should not be the same as that of the first subsample, the sampled data, or the overall data. Different data volumes correspond to different distribution balances. Changes in the distribution balance can be characterized by the change in the index value slope.

[0175] Figure 10 The index value slopes corresponding to the second sub-sampled data, the first sub-sampled data, and the sampled data are shown. The variation rule is determined based on the index value slopes.

[0176] For example, the first slope k0 from the second sub-sampled data to the first sub-sampled data is (0.42735-0.389) / (1-0)=0.03835, and the second slope k1 from the first sub-sampled data to the sampled data is (0.4564-0.42735) / (2-1)=0.03515. The rate of change of the slope is (0.03515-0.03835)=-0.0032.

[0177] Step 805: Determine the overall index value inclination rate according to the second index value inclination rate and the change rule of the index value inclination rate.

[0178] Taking the second sub-sample data as the benchmark, the upward merging is performed. After i merging times, the slope is ki = k0 – 0.0032*i, and the index value slope is g.i =g0+k0+k1+…+ki-1=g0+i*k0-(1+2+3+…+(i-1))*0.0032=g0+i*k0-(i-1)*i*0.0016.

[0179] If merging from the second sub-sampled data to the overall data is done nine times, then i = 9. Therefore, the overall index value slope of the overall data is obtained as g9 = 0.389 + 9 * 0.03835 - 72 * 0.0016 = 0.61895.

[0180] The sampled data is divided into w first sub-sampled data, each of which contains w second sub-sampled data. For the first sub-sampled data, the first index value slope of the first sub-sampled data can be determined based on the number of first sub-sampled scan lines corresponding to the first sub-sampled data and the distribution pattern of the first sub-sampled data. For the second sub-sampled data, the second index value slope of the second sub-sampled data can be determined based on the number of second sub-sampled scan lines corresponding to the second sub-sampled data and the distribution pattern of the second sub-sampled data. The variation pattern of the index value slope can then be summarized based on the second index value slope corresponding to the second sub-sampled data, the first index value slope corresponding to the first sub-sampled data, and the sampling index value slope corresponding to the sampled data. The overall index value slope can then be determined based on the second index value slope and the variation pattern of the index value slope, thereby improving the accuracy of the overall index value slope, and thereby improving the accuracy of the overall predicted number of scan lines for the overall data.

[0181] In step 304, the overall predicted number of scan lines of the overall data is determined according to the overall index value slope; the overall predicted number of scan lines is used to determine the predicted resource consumption of the command running in the overall data.

[0182] Specifically, it includes: determining the overall discrimination score of the overall data for the index based on the sampling discrimination score of the sampled data for the index; the discrimination score is used to characterize the number of different values ​​of the index in the data; determining the distribution pattern of the overall data based on the overall discrimination score and the data volume of the overall data; determining the overall predicted number of scan rows based on the distribution pattern of the overall data and the inclination rate of the overall index value.

[0183] By introducing the concept of a discrimination score, the overall discrimination score of the overall data for the index determined based on the sampling discrimination score of the sampled data for the index is relatively accurate. Therefore, determining the distribution pattern of the overall data based on the relatively accurate overall discrimination score and the overall data volume can improve the accuracy of the distribution pattern of the determined overall data. This in turn improves the accuracy of the determination of the second scan line number.

[0184] After considering the changes in the index value inclination rate according to the above method and determining a more accurate overall index value inclination rate, the predicted number of scan rows for the overall data can be reversed based on the definition of the index value inclination rate in the above text. In the above text, index value inclination y2 = number of scan rows / distribution pattern = number of scan rows / (total amount of data / number of distinctions). For example, in the above example, the total amount of data for the overall data is 17 million, and the overall number of distinctions for the overall data for a certain index, such as index 2, is 154. Overall index value inclination g9 = 0.389 + 9 * 0.03835 - 72 * 0.0016 = 0.61895. Therefore, the overall predicted number of scan rows = (total amount of data for the overall data / overall number of distinctions for index 2 for the overall data) * overall index value inclination rate = (17,000,000 / 154) * 0.61895 = 68,326 rows.

[0185] After actual verification, the actual number of scan rows obtained by running the SQL command on the overall data is about 100,000. This is not much different from the overall predicted number of scan rows of 68,326 obtained by the embodiment of this application. Figure 1 The method shown in the figure predicts 195,000 rows of overall data scanned. This is nearly double the actual number of scanned rows of 100,000. This demonstrates that the method provided in the embodiments of the present application can obtain a more accurate prediction of the overall number of scanned rows, which is also helpful for evaluating execution costs.

[0186] The corresponding execution cost is estimated by the following formula, i.e., the predicted resource consumption of the command in the overall data. Predicted resource consumption = (65500 / 775)*84.9 = 7175. Figure 1 The method shown determines the predicted resource consumption, and the predicted resource consumption obtained is 21378. It can be seen that there is still a large gap between this and the estimated cost of the present application.

[0187] Since the distribution balance degree of the overall data and the sampled data for the index corresponding to the command is different, there is currently a lack of a way to explore the changes in the distribution balance degree of the overall data and the sampled data respectively. The embodiment of the present application first executes the command in the sampled data of the table corresponding to the command, so that the accurate number of sampled scan rows can be obtained, and the distribution balance degree of the sampled data for the index corresponding to the command is quantified by the number of sampled scan rows and the distribution law of the sampled data, that is, the sampling index value inclination rate is obtained. Then, the overall index value inclination rate of the overall data can be determined, that is, the distribution balance degree of the overall data for the index corresponding to the command. In this way, the overall predicted number of scan rows determined according to the overall index value inclination rate will take into account the changes in the distribution balance degree of the overall data relative to the sampled data, so that the overall predicted number of scan rows is more accurate, which can improve the accuracy of determining the predicted resource consumption.

[0188] Based on the same technical concept, Figure 11 The structure of a device for determining predicted resource consumption of a database command provided by an embodiment of the present invention is exemplarily shown. The structure can execute the process of determining predicted resource consumption of a database command.

[0189] like Figure 11 As shown, the device specifically includes:

[0190] The determining unit 1101 is configured to:

[0191] Determine the table called in the database command;

[0192] Executing the command in the sampled data of the table to determine the number of sampled scan rows in which the command is executed in the sampled data; determining a sampling index value inclination rate of the sampled data based on a distribution pattern of the sampled data and the number of sampled scan rows; the index value inclination rate is used to represent a degree of distribution balance of the data with respect to the index corresponding to the command;

[0193] Determine the overall index value inclination rate of the overall data of the table according to the sampling index value inclination rate;

[0194] The overall predicted number of scan lines of the overall data is determined according to the overall index value slope; the overall predicted number of scan lines is used to determine the predicted resource consumption of the command running in the overall data.

[0195] In some embodiments, the sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data;

[0196] The determining unit 1101 is specifically configured to:

[0197] For any of the first sub-sampled data, determining a first index value inclination rate of the first sub-sampled data according to the number of first sub-sampled scan lines corresponding to the first sub-sampled data and a distribution rule of the first sub-sampled data;

[0198] For any second sub-sampled data, determining a second index value inclination rate of the second sub-sampled data according to the number of second sub-sampled scan lines corresponding to the second sub-sampled data and a distribution rule of the second sub-sampled data;

[0199] Determine a change rule of the index value slope from the second sub-sampled data to the sampled data according to the second index value slope, the first index value slope, and the sampled index value slope;

[0200] The overall index value inclination is determined according to the second index value inclination and a change rule of the index value inclination.

[0201] In some embodiments, the determining unit 1101 is specifically configured to:

[0202] Determine the overall discrimination score of the overall data for the index based on the sampling discrimination score of the sampled data for the index; the discrimination score is used to represent the number of different values ​​of the index in the data;

[0203] Determining the distribution pattern of the overall data according to the overall distinction number and the data volume of the overall data;

[0204] The overall predicted number of scan lines is determined according to the distribution pattern of the overall data and the overall index value inclination.

[0205] In some embodiments, the sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data;

[0206] The determining unit 1101 is specifically configured to:

[0207] determining a first non-overlapping rate of the sampled data for the index based on the w first sub-sampled data and the sampled data; the first non-overlapping rate is used to represent a proportion of the number of distinctions retained in the process of merging the w first sub-sampled data into the sampled data;

[0208] determining a second non-overlapping ratio of the first sub-sampled data with respect to the index based on the w second sub-sampled data and the first sub-sampled data corresponding to the w second sub-sampled data; the second non-overlapping ratio is used to represent a proportion of the number of distinctions retained in the process of merging the w second sub-sampled data into the first sub-sampled data;

[0209] Determine a variation rule of the non-overlapping rate from the first sub-sampled data to the sampled data according to the first non-overlapping rate and the second non-overlapping rate;

[0210] The overall discrimination number is determined according to the second non-overlapping rate, the variation rule of the non-overlapping rate, the data volume of the overall data, the sampling discrimination number and the data volume of the sampling data.

[0211] In some embodiments, the distribution pattern of the sampled data is the average amount of sampled data corresponding to each sampling zone score in the sampled data.

[0212] In some embodiments, the determining unit 1101 is specifically configured to:

[0213] The ratio of the sampling discrimination number of the sampled data with respect to the index to the sum of the first discrimination numbers of the w first sub-sampled data with respect to the index is used as the first non-overlapping rate.

[0214] In some embodiments, the determining unit 1101 is specifically configured to:

[0215] The entire data is sharded in the order of primary keys to obtain s shards; the amount of data in each shard is the same;

[0216] Divide the s shards into z groups according to the rule that each e shards constitutes a group;

[0217] For any of the z groups, select w 2 shards;

[0218] From the w 2 Select f data from the i-th shard in the shards as the w 2 A part of the i-th second sub-sampled data of the second sub-sampled data.

[0219] In some embodiments, the determining unit 1101 is specifically configured to:

[0220] The w 2 Any w second sub-sampled data among the w second sub-sampled data are merged to obtain the w first sub-sampled data; wherein the second sub-sampled data included in each first sub-sampled data is not repeated.

[0221] In some embodiments, the determining unit 1101 is specifically configured to:

[0222] The w first sub-sampled data are combined to obtain the sampled data.

[0223] Based on the same technical concept, the embodiment of the present application provides a computer device, such as Figure 12 As shown, it includes at least one processor 1201 and a memory 1202 connected to the at least one processor. The specific connection medium between the processor 1201 and the memory 1202 is not limited in the embodiment of the present application. Figure 10 For example, the processor 1201 and the memory 1202 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.

[0224] In an embodiment of the present application, the memory 1202 stores instructions that can be executed by at least one processor 1201. By executing the instructions stored in the memory 1202, the at least one processor 1201 can perform the steps of the above-mentioned method for determining the predicted resource consumption of the database command.

[0225] The processor 1201 is the control center of the computer device. It can connect various components of the computer device using various interfaces and lines. It determines the predicted resource consumption of database commands by running or executing instructions stored in the memory 1202 and accessing data stored in the memory 1202. In some embodiments, the processor 1201 may include one or more processing units. The processor 1201 may integrate an application processor and a modem processor. The application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 1201. In some embodiments, the processor 1201 and the memory 1202 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0226] The processor 1201 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0227] Memory 1202 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1202 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. Memory 1202 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1202 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0228] Based on the same technical concept, an embodiment of the present invention also provides a computer-readable storage medium, which stores a computer-executable program. The computer-executable program is used to enable a computer to execute any of the above methods for determining the predicted resource consumption of a database command.

[0229] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0230] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0231] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0232] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0233] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for determining predicted resource consumption of a database command, characterized in that: include: Determine the table called in the database command; Executing the command in the sampled data of the table to determine the number of sampled scan rows in which the command is executed in the sampled data; determining a sampling index value inclination rate of the sampled data based on a distribution pattern of the sampled data and the number of sampled scan rows; the sampling index value inclination rate is used to represent a degree of distribution balance of the data with respect to the index corresponding to the command; Determine the overall index value inclination rate of the overall data of the table according to the sampling index value inclination rate; The overall predicted number of scan lines of the overall data is determined according to the overall index value slope; the overall predicted number of scan lines is used to determine the predicted resource consumption of the command running in the overall data.

2. The method according to claim 1, wherein The sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data; Determining the overall index value inclination rate of the overall data of the table according to the sampling index value inclination rate includes: For any of the first sub-sampled data, determining a first index value inclination rate of the first sub-sampled data according to the number of first sub-sampled scan lines corresponding to the first sub-sampled data and a distribution rule of the first sub-sampled data; For any second sub-sampled data, determining a second index value inclination rate of the second sub-sampled data according to the number of second sub-sampled scan lines corresponding to the second sub-sampled data and a distribution rule of the second sub-sampled data; Determine a change rule of the index value slope from the second sub-sampled data to the sampled data according to the second index value slope, the first index value slope, and the sampled index value slope; The overall index value inclination is determined according to the second index value inclination and a change rule of the index value inclination.

3. The method according to claim 1, wherein Determining the overall predicted number of scan rows of the overall data according to the overall index value inclination includes: Determine the overall discrimination score of the overall data for the index based on the sampling discrimination score of the sampled data for the index; the discrimination score is used to represent the number of different values ​​of the index in the data; Determining the distribution pattern of the overall data according to the overall distinction number and the data volume of the overall data; The overall predicted number of scan lines is determined according to the distribution pattern of the overall data and the overall index value inclination.

4. The method according to claim 3, wherein The sampled data includes w first sub-sampled data; each first sub-sampled data includes w second sub-sampled data; Determining the overall discrimination score of the overall data for the index according to the sampling discrimination score of the sampled data for the index includes: determining a first non-overlapping rate of the sampled data for the index based on the w first sub-sampled data and the sampled data; the first non-overlapping rate is used to represent a proportion of the number of distinctions retained in the process of merging the w first sub-sampled data into the sampled data; determining a second non-overlapping ratio of the first sub-sampled data with respect to the index based on the w second sub-sampled data and the first sub-sampled data corresponding to the w second sub-sampled data; the second non-overlapping ratio is used to represent a proportion of the number of distinctions retained in the process of merging the w second sub-sampled data into the first sub-sampled data; Determine a variation rule of the non-overlapping rate from the first sub-sampled data to the sampled data according to the first non-overlapping rate and the second non-overlapping rate; The overall discrimination number is determined according to the second non-overlapping rate, the variation rule of the non-overlapping rate, the data volume of the overall data, the sampling discrimination number and the data volume of the sampling data.

5. The method according to claim 3, wherein The distribution rule of the sampled data is the data volume of the sampled data corresponding to the average number of sampling zones in the sampled data.

6. The method according to claim 4, wherein Determining a first non-overlapping rate of the sampled data for the index according to the w first sub-sampled data and the sampled data includes: The ratio of the sampling discrimination number of the sampled data with respect to the index to the sum of the first discrimination numbers of the w first sub-sampled data with respect to the index is used as the first non-overlapping rate.

7. The method according to claim 2 or 4, characterized in that The w first sub-sampled data contain w 2 The second sub-sample data is determined by the following method: 2 The i-th second sub-sample data among the second sub-sample data: The entire data is sharded in the order of primary keys to obtain s shards; the amount of data in each shard is the same; Divide the s shards into z groups according to the rule that each e shards constitutes a group; For any of the z groups, select w 2 shards; From the w 2 Select f data from the i-th shard in the shards as the w 2 A part of the i-th second sub-sampled data of the second sub-sampled data.

8. The method according to claim 7, wherein The w first sub-sample data are determined as follows: The w 2 Any w second sub-sampled data among the w second sub-sampled data are merged to obtain the w first sub-sampled data; wherein the second sub-sampled data included in each first sub-sampled data is not repeated.

9. The method according to claim 8, wherein The sampling data is determined as follows: The w first sub-sampled data are combined to obtain the sampled data.

10. A device for determining predicted resource consumption of a database command, characterized in that: include: Identify units for: Determine the table called in the database command; Executing the command in the sampled data of the table to determine the number of sampled scan rows in which the command is executed in the sampled data; determining a sampling index value inclination rate of the sampled data based on a distribution pattern of the sampled data and the number of sampled scan rows; the sampling index value inclination rate is used to represent a degree of distribution balance of the data with respect to the index corresponding to the command; Determine the overall index value inclination rate of the overall data of the table according to the sampling index value inclination rate; The overall predicted number of scan lines of the overall data is determined according to the overall index value slope; the overall predicted number of scan lines is used to determine the predicted resource consumption of the command running in the overall data.

11. A computing device, characterized in that include: memory for storing computer programs; A processor, configured to call a computer program stored in the memory and execute the method according to any one of claims 1 to 9 according to the obtained program.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the method according to any one of claims 1 to 9.

Images