Logical plan generation method and database system

Abstract

The present application provides a logical plan generation method and a database system, which can be applied to the technical field of computers. The method comprises: acquiring a bit array of a source table, computing an operation data volume of the source table on the basis of the bit array, and generating a logical plan on the basis of the operation data volume, wherein the operation data volume is a data volume, satisfying a screening condition, in the source table. Because the operation data volume can more accurately reflect the actual system resource requirements for executing a database operation, the logical plan generation method provided by the present application can generate a logical plan which is more rational and can better reflect actual requirements, so that when a physical plan corresponding to the logical plan is subsequently generated, the rationality of the physical plan is correspondingly improved, which is equivalent to improving the execution efficiency of user requests.

Description

Method for generating a logical plan and database system

[0001] The present application claims priority to the Chinese patent application No. 202410586958.7, filed on May 11, 2024, with the State Intellectual Property Office of China, and entitled "Method for generating a logical plan and database system", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of computer technology, and in particular to a method for generating a logical plan and a database system. BACKGROUND

[0003] The specific process of a database system responding to a single user request is as follows: first, a logical plan is generated according to the user request, the logical plan being used to describe the acquisition and processing methods of tables in the user request; then, a physical plan corresponding to the logical plan is generated, the physical plan being used to describe the system resource calling method for implementing the logical plan; finally, the physical plan is executed to obtain the request result. As can be seen, the specific content of the logical plan will affect the specific content of the physical plan, i.e., the system resource calling method, and the system resource calling method will affect the efficiency of completing the user request, so the rationality of the logical plan has an important influence on the execution efficiency of the user request.

[0004] For example, assuming that the user request involves tables A, B and C, in the case that the data volumes of tables A, B and C are quite different, the connection order of tables A, B and C described in the logical plan will affect the system resources required when the physical plan is executed. A good logical plan should select to use each row data in the table with smaller data volume to find the matching row data in the table with larger data volume, so as to minimize the data matching overhead when the physical plan is executed.

[0005] The existing logical plan determines the obtaining manner and processing manner of a table according to the statistical information (such as capacity, data volume) of the table in a single user request. With more and more tables involved in the user request and more and more complex relationships between the tables, the values of the above statistical information gradually increase, and the statistical information often contains invalid information. Therefore, determining the obtaining manner and processing manner of the table (i.e., the specific content of the logical plan) according to the statistical information of the table reduces the quality of the logical plan and makes the execution efficiency of the user request low. For example, the capacity of a table is 1 million bytes, and the actual occupied storage space is 50,000 bytes. Therefore, the invalid information is the difference (950,000 bytes) between the capacity of the table and the actual occupied storage space. If the table is processed according to the capacity, multiple scan steps will be generated in the logical plan, and each scan step is responsible for reading row data from a different index position in the table. Multiple computing nodes will be allocated in the subsequent physical plan to execute the multiple scan steps in the logical plan. Since the table only stores a small amount of data, this will make the multiple computing nodes perform empty operations, waste system resources, and the next step also needs to wait for the computing nodes to complete the empty operations, which seriously reduces the efficiency of completing the request result. SUMMARY

[0006] To solve the above technical problems, the present application provides a logical plan generation method and a database system, which can generate a more reasonable logical plan that can better reflect actual needs. This will correspondingly improve the rationality of the physical plan generated subsequently corresponding to the logical plan, which is equivalent to improving the execution efficiency of the user request.

[0007] In a first aspect, a logical plan generation method is provided. The method includes: first obtaining a bit array of a source table, then calculating an operation data volume of the source table according to the bit array, and finally generating a logical plan according to the operation data volume. The bit array includes multiple bits of Boolean values, and the bit array is used to indicate the mapping result between the data in the source table and the elements in the bit array. The operation data volume is the data volume of the data in the source table that meets a filtering condition or a connection condition. The logical plan is used to indicate the execution of a database operation according to a user request.

[0008] Since the statistical information of the source table describes the capacity or data volume of the source table, and the operation data volume of the source table describes the data volume of the source table that meets the filtering condition, the operation data volume of the source table is much smaller than the statistical information of the source table, and the operation data volume of the source table can more accurately reflect the actual system resource demand of the database operation, so the logical plan prepared based on the operation data volume is more reasonable and can better reflect the actual demand, which can improve the rationality of the physical plan generated based on the logical plan, and equivalently improve the execution efficiency of the user request. On the other hand, the operation data volume of the source table is calculated based on the mapping result of the data in the source table in the bit array, compared with the way of actually performing the database operation on the source table to obtain the operation data volume of the source table, the mapping way in the present technical solution can quickly calculate the operation data volume of the source table, avoiding the indirect negative impact of long operation data volume acquisition time on the execution efficiency of the user request.

[0009] In some possible implementation manners, before the bit array of the source table is obtained, the method further includes: filtering the first column set associated data in the source table according to the filtering condition, converting the first column set associated data into a first index, and then setting the elements in the bit array indicated by the first index to 1 respectively, so as to obtain the bit array of the source table.

[0010] In the above scheme, after the first column set associated data related to the filtering condition is obtained from the source table, compared with the way of directly storing the first column set associated data and then applying it to the subsequent operation data volume calculation, since one bit in the bit array can represent the existence or nonexistence of one data, the use of the bit array to store the first column set associated data can greatly reduce the storage space occupation. On the other hand, the operation of the bit array is at the bit level, supports bit operation, and supports parallel computing, so when the amount of the first column set associated data is large, the subsequent calculation of the operation data volume by using the mapping result of the large amount of first column set associated data in the bit array can take advantage of the parallel computing to improve the calculation efficiency.

[0011] In some possible implementation manners, the conversion of the first column set associated data into the first index includes: calculating the first column set associated data by using a hash function to obtain a binary calculation result, converting the binary calculation result into a decimal value, and taking the decimal value as the first index.

[0012] In the above scheme, the first index can be quickly obtained by using the hash function to simply convert the first column set associated data, so as to improve the calculation speed of the operation data volume.

[0013] In some possible implementation manners, in the case where the screening condition is the filter condition, the operation data amount of the source table according to the bit array includes: converting the filter condition into a second index, and calculating the number of times that an element in the bit array indicated by the second index is set to 1, so as to obtain the operation data amount of the source table.

[0014] In the scheme, the data related to the filter condition in the source table is mapped into the bit array, the filter condition is also mapped into the same bit array, and the operation data amount of the source table is determined according to the relationship between the mapping result of the data related to the filter condition in the source table and the mapping result of the filter condition, so that the operation data amount can be quickly obtained, and the operation data amount obtained by calculation is accurate.

[0015] In some possible implementation manners, the converting the filter condition into the second index includes: calculating the filter condition by using a hash function to obtain a binary calculation result, converting the binary calculation result into a decimal numerical value, and taking the decimal numerical value as the second index.

[0016] In the scheme, the filter condition is simply converted by using the hash function, so that the second index can be quickly obtained, and the calculation speed of the operation data amount is improved.

[0017] In some possible implementation manners, the calculating the number of times that the element in the bit array indicated by the second index is set to 1, so as to obtain the operation data amount of the source table includes: taking a minimum value or a maximum value of the number of times that the element in the bit array indicated by the second index is set to 1 as the operation data amount, or calculating an average value of the number of times that the element in the bit array indicated by the second index is set to 1 as the operation data amount.

[0018] In the scheme, multiple ways of calculating the operation data amount are provided in the case where the screening condition is the filter condition, so that different calculation environments and calculation requirements can be flexibly adapted, and the universality of the technical scheme is also embodied.

[0019] In some possible implementation manners, in the case where the screening condition is the join condition, the operation data amount of the source table according to the bit array includes: screening a second column set in an association table according to the join condition, converting second column set association data into a second index, and calculating the number of times that an element in the bit array indicated by the second index is set to 1, so as to obtain the operation data amount of the source table. The association table is another table that is connected with the source table according to the join condition.

[0020] In the scheme, the data related to the connection condition in the source table is mapped into the bit array, the data related to the connection condition in the association table is also mapped into the same bit array, and the operation data amount of the source table is determined according to the relationship between the mapping result of the data related to the filter condition in the source table and the mapping result of the data related to the filter condition in the association table, so that the operation data amount can be quickly obtained, and the operation data amount obtained by calculation is accurate.

[0021] In some possible implementation manners, the operation data amount of the source table is obtained by calculating the number of times that the element in the bit array indicated by the second index is set to 1, including: selecting a minimum value or a maximum value from the number of times that the element in the bit array indicated by the second index is set to 1, or calculating an average value of the number of times that the element in the bit array indicated by the second index is set to 1; and taking the sum of the plurality of minimum values as the operation data amount, or taking the sum of the plurality of maximum values as the operation data amount, or taking the sum of the plurality of average values as the operation data amount.

[0022] In the scheme, multiple ways of calculating the operation data amount when the screening condition is the connection condition are provided, so that different computing environments and computing requirements can be flexibly adapted, and the universality of the technical scheme is also embodied.

[0023] In a second aspect, a database system is provided, which includes various modules for executing the fault processing method in the first aspect or any possible implementation manner of the first aspect.

[0024] In a third aspect, a chip system is provided, which includes a processor and a power supply circuit for supplying power to the processor, and the processor is configured to execute the method according to any one of the first aspect.

[0025] In a fourth aspect, a computing device is provided, which includes a processor and a memory for storing instructions, and the processor is configured to execute the instructions, so as to implement the method according to any one of the first aspect.

[0026] In a fifth aspect, a computing device cluster is provided, which includes at least one computing device, and each computing device includes a processor and a memory.

[0027] The processor of the at least one computing device is configured to execute the instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of the first aspect.

[0028] In a sixth aspect, a computer program product including instructions is provided, which, when executed by a computing device, causes the computing device to execute the method according to any one of the first aspect.

[0029] In a seventh aspect, a computer-readable storage medium is provided, and the computer-readable storage medium comprises computer program instructions, when the computer program instructions are executed by a computing device, the computing device performs the method of any one of the first aspect.

[0030] On the basis of the implementation manners of the above aspects, the application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0031] FIG. 1 is a structural schematic diagram of a database system provided by the application;

[0032] FIG. 2 is a flow schematic diagram of a logical plan generation method provided by the application;

[0033] FIG. 3A is an initialization result schematic diagram of a bit array of a source table provided by the application;

[0034] FIG. 3B is a schematic diagram of a bit array of a source table provided by the application;

[0035] FIG. 4A is an initialization result schematic diagram of a bit array of another source table provided by the application;

[0036] FIG. 4B is a schematic diagram of a bit array of another source table provided by the application;

[0037] FIG. 5 is a structural schematic diagram of a computing device provided by the application;

[0038] FIG. 6 is a structural schematic diagram of a computing device cluster provided by the application;

[0039] FIG. 7 is a structural schematic diagram of another computing device cluster provided by the application. DETAILED DESCRIPTION

[0040] For ease of understanding, before introducing the technical solution to be protected by the application, the concepts and relationships of user request, logical plan and physical plan are briefly introduced.

[0041] I. User request

[0042] The user request can be a function request associated with a business (such as annual report generation). To implement the function request associated with the business, it is usually necessary to use the data stored in the database system. Therefore, the user request can be a structured query language (SQL) statement, so that the data in the database system can be obtained, and the function request associated with the business can be completed based on the obtained data. Further, the operation object of the user request is usually a table stored in the database system. The user can query, insert, update or delete the table in the database system through the SQL statement.

[0043] Two SQL statements are provided below as examples to illustrate user requests.

[0044] SQL statement S1: for querying a single table, specifically "SELECT * FROM employees WHERE dept_name='Sales'". The goal of SQL statement S1 is to query the employee table (hereinafter referred to as table employees) and return the row data of employees with the department name (dept_name) of "Sales" in table employees. The employee table is shown in Table 1 below. For the sake of brevity of the description, Table 1 only shows 5 row data.

[0045] Table 1

[0046] Table 1 (table employees) records the personal information of employees, including employee number (emp_no), employee name (first_name, last_name), hire date (hire_date), salary (salary), salary start date (from_date), salary settlement date (to_date), department number (dept_no), and department name (dept_name), etc.

[0047] SQL statement S2: for querying multiple tables, specifically "SELECT orders.order_id, customers.customer_name FROM orders JOIN customers ON orders.customer_id=customers.customer_id JOIN products ON orders.product_id=products.product_id WHERE orders.order_date>='2022-01-01'". The goal of SQL statement S2 is to query the order table (hereinafter referred to as table orders), the customer table (hereinafter referred to as table customers), and the product table (hereinafter referred to as table products), and return the order number (order_id) in table orders and the customer name (customer_name) in table customers for orders with an order date (order_date) on or after January 1, 2022. The order table, customer table, and product table are shown below. For the sake of brevity of the description, the order table, customer table, and product table each only show the specific content of 5 row data.

[0048] Table 2

[0049] Table 2 (table orders) records order information, including order number (order id), order date (order date), customer number (customer id), product number (product id), and product quantity (product quantity), and so on.

[0050] Table 3

[0051] Table 3 (table customers) records customer information, including customer number (customer id), customer name (customer name), customer city (customer city), and customer gender (customer gender), and so on.

[0052] Table 4

[0053] Table 4 (table products) records product information, including product number (product id), product name (product name), and product price (product price), and so on.

[0054] II. Logical plan

[0055] The logical plan is used to indicate the execution of the database operation according to the user request. Specifically, the logical plan is used to indicate the acquisition and processing manner of the data associated with the user request, usually in the form of logical operators and execution order. The following takes the SQL statement S1 and the SQL statement S2 in the above user request as an example to describe the logical plan.

[0056] The logical plan L1 generated according to the above SQL statement S1 is specifically as follows:

[0057] Step 1 - scan: read each row data of the table employees by full table scan.

[0058] Step 2 - filter: match the department name (dept name) of each row data in the table employees with 'Sales', and keep the row data whose department name (dept name) is 'Sales'.

[0059] Step 3 - return: the row data kept in step 2.

[0060] The logical plan L2 generated according to the above SQL statement S2 is specifically as follows:

[0061] Step 1 - Scan: Read each row data of table orders by full table scan.

[0062] Step 2 - Scan: Read each row data of table customers by full table scan.

[0063] Step 3 - Join: Match the customer_id of each row data of table orders with the customer_id of each row data of table customers, associate the row data of table orders and table customers with the same customer_id.

[0064] Step 4 - Scan: Read each row data of table products by full table scan.

[0065] Step 5 - Join: Match the product_id of each row data of table orders with the product_id of each row data of table products, associate the row data of table orders and table products with the same product_id.

[0066] Step 6 - Filter: Match the row data of table orders associated in Step 3 with the row data of table orders associated in Step 5, keep the same row data.

[0067] Step 7 - Filter: Match the order_date of the row data kept in Step 6 with '2022-01-01', keep the row data whose order_date is on or after January 1, 2022.

[0068] Step 8 - Return: The order_id of the row data kept in Step 7, and the customer_name of the row data of table customers associated with the row data kept in Step 7.

[0069] From the logical plan L1 and the logical plan L2, we can know that:

[0070] (1) The operations in the logical plan that involve changes in data volume are filtering operations or joining operations.

[0071] Filter operation, used to filter data from a table according to filter conditions. The filter conditions are specified using a WHERE clause. For example, step 2—filter of logical plan L1 is to filter out rows of table employees with dept_name = 'Sales'. Therefore, the data volume after performing the filter operation is usually less than the actual data record number of table employees (i.e. the data volume of table employees). Generally, the data volume is represented by the number of rows.

[0072] Join operation, used to join data from two tables according to join conditions. The join conditions are specified using an ON clause in a JOIN clause. For example, step 3—join of logical plan L2 is to join rows of table orders and table customers with the same customer_id. Therefore, the data volume after performing the join operation is usually less than the original data volume of table orders.

[0073] (2) A step only operates on a single table or two tables. For example, step 2 of logical plan L1 only performs a filter operation on table employees. Step 3 of logical plan L2 only performs a join operation on table orders and table customers.

[0074] (3) Any two steps can have a dependency relationship or be independent of each other. For example, the join operation of step 3 of logical plan L2 depends on the scan operation of step 1 and the scan operation of step 2, while the join operation of step 3 is independent of the scan operation of step 4, which means that step 3 can be executed before step 4 or after step 4.

[0075] III. Physical plan

[0076] The physical plan is used to indicate the system resource invocation manner for implementing the logical plan, and generally includes access path, index usage, connection order, etc. The following continues to take the above-mentioned (2) logical plan L1 and logical plan L2 in the logical plan as an example to describe the physical plan.

[0077] The physical plan P1 generated according to logical plan L1 is specifically as follows:

[0078] Step 1—corresponding to step 1 of logical plan L1, specifically: determine the data node storing table employees.

[0079] Step 2—corresponding to step 1 of logical plan L1, specifically: perform a full table scan on table employees in the data node to obtain each row data of table employees.

[0080] Step 3 - corresponds to Step 2 of logical plan LI, specifically: check whether the department name (dept_name) of each row data is 'Sales'. If the department name (dept_name) is 'Sales', keep the row data; otherwise, discard the row data.

[0081] Step 4 - corresponds to Step 3 of logical plan LI, specifically: return the row data kept in Step 3.

[0082] Therefore, by executing physical plan PI, the requested result Rl can be obtained, as shown in Table 5 below:

[0083] Table 5

[0084] The physical plan P2 generated according to the logical plan L2 is specifically:

[0085] Step 1 (corresponds to Step 3 of logical plan L2): determine the data node storing the table orders and the table customers.

[0086] Step 2 (corresponds to Step 1 of logical plan L2): perform a full table scan on the table orders in the data node of Step 1 to obtain each row data of the table orders.

[0087] Step 3 (corresponds to Step 2 of logical plan L2): perform a full table scan on the table customers in the data node of Step 1 to obtain each row data of the table customers.

[0088] Step 4 (corresponds to Step 3 of logical plan L2): check whether the customer number (customer_id) of each row data in the table orders exists in the table customers. If it exists, associate the row data with the row data in the table customers having the same customer number (customer_id); otherwise, discard the row data.

[0089] Step 5 (corresponds to Step 5 of logical plan L2): determine the data node storing the table orders and the table products.

[0090] Step 6 (corresponds to Step 4 of logical plan L2): perform a full table scan on the table products in the data node of Step 6 to obtain each row data of the table products.

[0091] Step 7 (corresponds to Step 5 of logical plan L2): perform a full table scan on the table orders in the data node of Step 6 to obtain each row data of the table orders.

[0092] Step 8 (corresponding to Step 5 of the logical plan L2): Check whether the product ID (product_id) of each row data in the table orders exists in the table products. If it exists, associate the row data with the row data in the table products that has the same product ID (product_id); otherwise, discard the row data.

[0093] Step 9 (corresponding to Step 6 of the logical plan L2): Check whether the row data in the table orders associated in Step 4 is the same as the row data in the table orders associated in Step 8. If it is the same, keep the row data; otherwise, discard the row data.

[0094] Step 10 (corresponding to Step 7 of the logical plan L2): Check whether the order date (order_date) of the row data kept in Step 9 is on or after January 1, 2022. If the order date (order_date) is on or after January 1, 2022, keep the row data; otherwise, discard the row data.

[0095] Step 11 (corresponding to Step 8 of the logical plan L2): Return the order ID (order_id) of the row data kept in Step 10, and the customer name (customer_name) of the row data in the table customers associated with the row data kept in Step 10.

[0096] Therefore, by executing the physical plan P2, the request result R2 can be obtained, as shown in the following Table 6:

[0097] Table 6

[0098] As can be seen from the physical plan P1 and the physical plan P2, the physical plan is generated according to the logical plan, and thus the rationality of the logical plan directly determines the rationality of the physical plan.

[0099] In the process of optimizing the logical plan, it can be found that the operation data volume of the source table is an important consideration for optimizing the logical plan. The source table is a table (such as the table employees in the aforementioned (1) user request) in the logical plan that is subject to a database operation (such as a filtering operation or a connection operation), and usually contains data that needs to be queried, extracted, converted, or loaded. The operation data volume of the source table is the data volume of the data in the source table that meets the screening condition (such as the filtering condition corresponding to the filtering operation, or the connection condition corresponding to the connection operation).

[0100] The following will take the amount of data meeting the join condition (hereinafter referred to as the associated data amount) obtained by performing a join operation on two source tables as an example of the operation data amount of the source table, and specifically describe the role of the operation data amount of the source table in the optimization process of the logical plan through three scenarios.

[0101] Scenario 1: The associated data amount determines the join algorithm of two source tables in the logical plan. When the associated data amount is large, a large amount of data matching and comparison needs to be performed, and therefore a join algorithm with high efficiency, low memory consumption, and supporting index needs to be selected. Among them, the data amount of the source table is the actual number of data records stored by the source table, which is usually represented by the number of rows.

[0102] Scenario 2: The associated data amount determines the index mode of two source tables in the logical plan. When the associated data amount is large, the data matching can be performed in a mode of not using index or full table scan to avoid a large number of input / output (I / O) operations caused by index. When the associated data amount is small, the data matching can be performed in a mode of using index.

[0103] Scenario 3: The associated data amount determines the join order of two source tables in the logical plan. When the associated data amount is large, the data matching is performed in a mode of using each row data in the source table with smaller data amount to find the matching row data in the source table with larger data amount, so as to reduce the overhead of data matching.

[0104] From the above three scenarios, it can be seen that the size of the associated data amount will affect the optimization of the logical plan. Considering the size of the associated data amount when optimizing the logical plan can make the optimized logical plan have higher query efficiency and performance. This means that the accuracy of the operation data amount of the source table is crucial to the reasonableness of the logical plan. The more accurate the operation data amount of the source table is calculated, the more reasonable the logical plan is; the greater the deviation of the operation data amount of the source table is calculated, the less reasonable the logical plan is.

[0105] In order to solve the problem that the way of generating a logical plan according to statistical information of a source table results in poor quality of the logical plan, and thus low efficiency of execution of a user request, the present application provides a database system which can obtain a bit array of the source table, and then calculate operation data volume of the source table according to the bit array of the source table, and further generate a logical plan according to the operation data volume of the source table. The bit array is used to indicate a mapping result between data in the source table and elements in the bit array. The operation data volume is the data volume of data in the source table that meets a filtering condition. The database system can generate a logical plan according to the operation data volume which accurately reflects actual system resource demand in execution of a database operation, so that the logical plan generated by the database system is more reasonable and better reflects actual demand, which can improve the rationality of a physical plan corresponding to the logical plan, and equivalently improve the efficiency of execution of a user request.

[0106] Referring to FIG. 1, FIG. 1 is a structural schematic diagram of a database system provided by the present application. As shown in FIG. 1, the database system comprises a client 100, a database management system 200, a computing node 300, and a data node 400. The client 100 and the database management system 200, the database management system 200 and the computing node 300, and the computing node 300 and the data node 400 can communicate through wired or wireless means.

[0107] The client 100 is configured to receive a user request input by a user, input the user request into the database management system 200, and receive a request result obtained by the database management system 200 according to the user request.

[0108] The database management system 200 is configured to generate an access instruction according to the user request, send the access instruction to the computing node 300, and receive a request result obtained by the computing node 300 according to the access instruction.

[0109] The computing node 300 is configured to process data in the data node 400 according to the access instruction, and thus obtain the request result.

[0110] The data node 400 is configured to store data. Generally, the data is stored in the form of a table in the data node 400. For example, the data node 400 stores the table employees of the above-mentioned one, and the SQL statement S1 in the user request.

[0111] In some possible implementation manners, the database system can be a centralized database or a distributed database. The centralized database refers to a database system in which all data is stored in a single data node. The distributed database refers to a database system in which data is stored in multiple data nodes, and data sharing and processing are performed through network connection.

[0112] ​When the database system is a centralized database, the client 100, the database management system 200, the computing node 300, and the data node 400 can be deployed on the same computing device. The computing device is an electronic device for computing, processing, and storing data, including a server, a supercomputer, a personal computer, a workstation, a mobile device, and the like.

[0113] When the database system is a distributed database, the client 100, the database management system 200, the computing node 300, and the data node 400 can be deployed on different computing devices. Specifically, at least two of the client 100, the database management system 200, the computing node 300, and the data node 400 are deployed on different computing devices in the same computing device cluster, or are deployed on different computing devices in different computing clusters. The computing device cluster can include a plurality of computing devices as described above. The specific deployment can be determined according to the actual application scenario, and the present application does not make specific limitations.

[0114] In some possible implementations, the database management system 200 includes a plurality of units. For example, in FIG. 1, the database management system 200 includes an obtaining unit 210, a logic unit 220, a physical unit 230, and an execution unit 240. The logic unit 220 includes a mapping unit 221, a computing unit 222, and a generating unit 223.

[0115] The obtaining unit 210, the logic unit 220, the physical unit 230, the execution unit 240, the mapping unit 221, the computing unit 222, and the generating unit 223 can be implemented by software or by hardware. For example, the implementation of the obtaining unit 210 is described below. Similarly, the implementation of the logic unit 220, the physical unit 230, the execution unit 240, the mapping unit 221, the computing unit 222, and the generating unit 223 can be implemented by referring to the implementation of the obtaining unit 210.

[0116] As an example of the software functional unit, the obtaining unit 210 can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the obtaining unit 210 can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers for running the code can be distributed in the same region, or can be distributed in different regions. Further, the multiple hosts / virtual machines / containers for running the code can be distributed in the same availability zone (AZ), or can be distributed in different AZs, each of which includes a data center or multiple data centers with similar geographical locations. Generally, one region can include multiple AZs.

[0117] Similarly, the multiple hosts / virtual machines / containers for running the code can be distributed in the same virtual private cloud (VPC), or can be distributed in multiple VPCs. Generally, one VPC is set in one region, and communication between two VPCs in the same region and between VPCs in different regions needs to be set in each VPC to set a communication gateway, and the interconnection between VPCs is realized through the communication gateway.

[0118] As an example of the hardware functional unit, the obtaining unit 210 can include at least one computing device, such as a server, etc. Alternatively, the obtaining unit 210 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. The PLD can be implemented by a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0119] The multiple computing devices included in the obtaining unit 210 can be distributed in the same region or in different regions. The multiple computing devices included in the obtaining unit 210 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the obtaining unit 210 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0120] The functions of each unit in the database management system 200 will be described below.

[0121] The obtaining unit 210 is configured to receive a user request input by a user through the client 100 and input the user request to the logical unit 220.

[0122] The logical unit 220 is configured to generate a logical plan according to the user request and input the logical plan to the physical unit 230. The unit in the logical unit 220 for generating the logical plan according to the user request is as follows:

[0123] The mapping unit 221 is configured to map the data in the source table in the user request to obtain a bit array of the source table. The source table is a table in the user request to be executed by a database operation. The bit array of the source table is composed of multiple bits of Boolean values. The process of mapping the data in the source table in the user request to obtain the bit array of the source table can be referred to the execution process of the database management system mapping the data in the source table in the user request to obtain the bit array of the source table in step S102 of the logical plan generation method of FIG. 2 described below.

[0124] The computing unit 222 is configured to calculate the operation data volume of the source table according to the bit array of the source table. The operation data volume of the source table is the data volume of the source table that meets the filtering condition. The process of calculating the operation data volume of the source table according to the bit array of the source table can be referred to the execution process of the database management system calculating the operation data volume of the source table according to the bit array of the source table in step S103 of the logical plan generation method of FIG. 2 described below.

[0125] The generating unit 223 is configured to generate a logical plan according to the operation data volume of the source table. The logical plan is configured to instruct to execute a database operation according to the user request. The process of generating the logical plan according to the operation data volume of the source table can be referred to the execution process of the database management system generating the logical plan according to the operation data volume of the source table in step S104 of the logical plan generation method of FIG. 2 described below.

[0126] The physical unit 230 is configured to generate a physical plan corresponding to the logical plan, and send the physical plan to the execution unit 240. The process can refer to the execution process of the database management system generating the physical plan according to the logical plan in step S105 of the logical plan generation method of FIG. 2.

[0127] The execution unit 240 is configured to execute the physical plan to obtain the request result. The execution unit 240 is specifically configured to convert the physical plan into an access instruction, send the access instruction to the computing node 300, and receive the request result obtained by the computing node 300 according to the access instruction. The process can refer to the execution process of the database management system executing the physical plan to obtain the request result in step S106 of the logical plan generation method of FIG. 2.

[0128] The execution unit 240 is further configured to send the request result to the client 100.

[0129] It should be understood that the units in the above-mentioned database management system 200 are exemplarily described by taking the obtaining unit 210 to the execution unit 240 of the database management system 200 in FIG. 1 as an example. In actual application, the database management system 200 can further include more or fewer units, which are not specifically limited in the present application.

[0130] It should be understood that the above-mentioned database system in FIG. 1 is exemplarily described by taking one, three, and four as examples of the client 100, the computing node 300, and the data node 400 respectively communicating with the database management system 200, and each of the data nodes 400 communicating with each of the computing nodes 300. In actual application, the number of the client 100, the computing node 300, and the data node 400 in the database system can be one or more, which are not specifically limited in the present application.

[0131] In summary, the database management system 200 provided in the present application can first obtain the bit array of the source table, then calculate the operation data volume of the source table according to the bit array, and finally generate the logical plan according to the operation data volume. Since the statistical information of the source table describes the capacity or data volume of the source table, and the operation data volume of the source table describes the data volume of the source table meeting the filtering condition, the operation data volume of the source table is much smaller than the statistical information of the source table, and the operation data volume of the source table can more accurately reflect the actual system resource demand of executing the database operation. Therefore, the logical plan formulated by the database management system 200 based on the operation data volume is more reasonable and can better reflect the actual demand, which can improve the rationality of the physical plan generated according to the logical plan, and equivalently improve the execution efficiency of the user request.

[0132] The generation method of the logical plan provided in the present application will be introduced below. In order to facilitate the understanding of the method, after the method is introduced, the execution process of the method will be specifically introduced through two examples, including (1) Example 1 and (2) Example 2, so as to illustrate the universality of the method.

[0133] Referring to FIG. 2, FIG. 2 is a flowchart of a generation method of a logical plan provided in the present application. As shown in FIG. 2, the generation method of the logical plan provided in the present application comprises:

[0134] S101: The client sends a user request to the database management system.

[0135] Correspondingly, the acquisition unit in the database management system receives the user request from the client. The client can be the client 100 in FIG. 1, the database management system can be the database management system 200 in FIG. 1, and the acquisition unit can be the acquisition unit 210 in the database management system 200 in FIG. 1.

[0136] The user request can be the aforementioned SQL statement S1 or SQL statement S2 in the user request. The client is used to receive the user request input by the user. The client can be deployed on a computing device or a terminal device. The computing device is an electronic device used for computing, processing and storing data, including servers, supercomputers, personal computers, workstations, mobile devices, etc. The terminal device is an electronic device used for accessing the computing device, including personal computers, smart phones, palm processing devices, tablet computers, mobile notebooks, integrated palm devices, smart conference devices, smart advertising devices, etc. The database management system is used to obtain a request result according to the user request. The database management system can be deployed on a computing device.

[0137] S102: The mapping unit in the database management system maps the data in the source table in the user request to obtain a bit array of the source table.

[0138] The mapping unit can be the mapping unit 221 in the logical unit 220 in the database management system 200 in FIG. 1.

[0139] The source table is a table in the user request to be executed by the database operation. The bit array (also known as binary bit array) of the source table is composed of multiple bits of Boolean values, such as 01110100010010.

[0140] In some possible implementations, the database management system maps the data in the source table in the user request to obtain a bit array of the source table, including: determining the number of Boolean values in the bit array according to statistical information of the source table, initializing the bit array, mapping the data in the source table to the bit array, and taking the mapping result as the bit array of the source table.

[0141] The statistical information of the source table is used to describe the quantity characteristics of the source table, for example, can be the capacity or the data amount of the source table. The capacity of the source table is the storage space occupied by the source table, and is usually expressed in storage units such as bytes, gigabytes or gigabits. The data amount of the source table is the number of data records actually stored by the source table, and is usually expressed in the number of rows. The calculation process of the statistical information of the source table can be realized by querying the system table or the data dictionary. If the statistical information is the data amount, the COUNT aggregation function can also be used to count the data in the source table, so as to obtain the data amount of the source table.

[0142] The above-mentioned database management system can refer to the following determination mode 1 and determination mode 2 to determine the number of Boolean values in the bit array according to the statistical information of the source table.

[0143] Determination mode 1: a hash function is used to determine the number of Boolean values in the bit array based on the statistical information of the source table.

[0144] Determination mode 2: a bloom filter is used to determine the number of Boolean values in the bit array based on the statistical information of the source table.

[0145] In determination mode 1 of step S102, the hash function is a function of mapping data of any length to a hash value of fixed length. The hash function can be, for example, message-digest algorithm 5 (MD5), secure hash algorithm 1 (SHA-1) or SHA-256, etc.

[0146] In some possible implementation manners, the number of the Boolean values in the bit array is determined based on the statistical information of the source table by using the hash function, specifically, converting the capacity or data volume of the source table in decimal into a first value in binary, inputting the first value into the hash function, taking the binary value output by the hash function as a second value, converting the second value into a third value in decimal, and taking the third value as the number of the Boolean values in the bit array. The process of inputting the first value into the hash function to obtain the second value can refer to the process of converting input data into output data by using MD5, SHA-1 or SHA-256, which will not be described herein again for the sake of brevity and conciseness of the description. For the ease of understanding, the specific application of the determination manner can refer to the specific process of applying step S103 in (I) Example 1 below or the specific process of applying step S103 in (II) Example 2 below.

[0147] In the determination manner 2 of step S102, the Bloom filter is a probabilistic data structure, including a bit array and a series of hash functions.

[0148] In some possible implementation manners, the number of the Boolean values in the bit array is determined based on the statistical information of the source table by using the Bloom filter, specifically, calculating the number of the Boolean values in the bit array based on the capacity (or data volume) of the source table and the false positive rate threshold of the Bloom filter. The calculation formula of the number of the Boolean values in the bit array is as follows:

[0149] wherein m represents the number of the Boolean values in the bit array, n represents the statistical information of the source table, for example, the capacity or data volume of the source table, ln() represents the logarithm with the natural constant e as the base, and ε0 represents the false positive rate threshold, which is determined by the user.

[0150] It should be understood that the determination manner 1 and the determination manner 2 of the number of the Boolean values in the bit array in step S102 are only examples and are not limited herein. In actual applications, any implementation manner that can determine the number of the Boolean values in the bit array based on the statistical information of the source table is within the protection scope of the present application.

[0151] After the database management system determines the number of the Boolean values in the bit array, the bit array is initialized so that the initial values of all the Boolean values in the bit array are 0, for example, the bit array containing fourteen Boolean values is initialized, and the result is 00000000000000.

[0152] The database management system maps the data in the source table into a bit array, and takes the mapping result as the bit array of the source table. Specifically, the first column set associated data is converted into a first index one by one according to the filtering condition or the connection condition, and the elements in the bit array indicated by the first index are set to 1 respectively until all the first column set associated data is converted, and the setting result is taken as the bit array of the source table. For the convenience of understanding, the filtering condition and the connection condition will be taken as examples of the filtering condition in the following description. When the filtering condition is the filtering condition, the specific application of obtaining the bit array of the source table according to the filtering condition can be referred to the specific process of applying step S104 in example 1 below. When the filtering condition is the connection condition, the specific application of obtaining the bit array of the source table according to the connection condition can be referred to the specific process of applying step S104 in example 2 below.

[0153] Optionally, the more the number of Boolean values in the bit array, the more the number of bit positions available in the bit array, which can reduce the probability of different data being mapped to the same bit position, thereby reducing the occurrence of repetition and conflict problems.

[0154] The first index of the first column set associated data is obtained by converting the first column set associated data using one or more hash functions. The first index can be quickly obtained by converting the first column set associated data using a hash function, thereby improving the speed of the entire calculation process of the operation data. One hash function can calculate one index number in the first index of the first column set associated data. Therefore, the number of index numbers in the first index of the first column set associated data is the same as the number of hash functions used for conversion. When multiple hash functions are used to convert the first column set associated data, the hash functions are used to calculate the first column set associated data respectively to obtain binary calculation results, and each binary calculation result is converted into a decimal value, and each decimal value is taken as an index number in the first index of the first column set associated data.

[0155] Optionally, the more the number of hash functions used for conversion, the greater the diversity and randomness of the first index obtained by different data through the hash functions, thereby reducing the repetition and conflict problems of the first index of different data. In addition, even if two data use the same hash function to obtain the same first index, the first index obtained by the two data using another hash function will be different, which can also effectively reduce the repetition and conflict problems of the first index representing the data.

[0156] In a specific implementation, the number of hash functions used for conversion is determined based on the number of Boolean values in the bit array. The number of Boolean values in the bit array is obtained by using the determination method 2 in step S102. The number of hash functions used for conversion is determined based on the false positive rate calculation formula of the Bloom filter, the relationship between the number of hash functions used for conversion and the number of Boolean values in the bit array is determined, and then the number of hash functions used for conversion is calculated according to the specific value of the number of Boolean values in the bit array.

[0157] The false positive rate calculation formula of the Bloom filter is:

[0158] In the implementation method 2 in step S101, the length of the initial bit array is calculated by the formula:

[0159] Therefore, in the case of ε = ε0, the relationship between the number of hash functions and the length of the initial bit array is:

[0160] Wherein, ε represents the false positive rate; e represents the natural constant; k represents the number of hash functions; n represents the statistical information of the source table, such as the capacity or data volume of the source table; m represents the length of the initial bit array; ln() represents the logarithm with the natural constant e as the base; ε0 represents the false positive rate threshold.

[0161] S103: The computing unit in the database management system calculates the operation data volume of the source table according to the bit array of the source table.

[0162] The computing unit can be the computing unit 222 in the logical unit 220 in the database management system 200 in FIG. 1.

[0163] On the basis of mapping the data in the source table to the bit array, in order to judge the relationship between the data mapped to the bit array and the filtering condition (such as whether the data mapped to the bit array meets the filtering condition, or whether the data mapped to the bit array and the data in the association table meet the connection condition) in the same dimension, the filtering condition also needs to be mapped to the bit array of the source table. The association table is another table connected with the source table according to the connection condition.

[0164] When the filtering condition is different, the way to calculate the operation data volume of the source table according to the bit array of the source table is also different.

[0165] When the filtering condition is a filtering condition, the operation data amount of the source table is calculated according to the bit array of the source table, specifically: the filtering condition is converted into a second index, and then the number of times that the element in the bit array indicated by the second index is set to 1 is calculated, and the calculation result is taken as the operation data amount of the source table.

[0166] The hash function used to convert the filtering condition into the second index is the same as the hash function used to convert the first column set associated data into the first index in the above-mentioned step S102, so the filtering condition can also be converted simply, so as to quickly obtain the second index, and further improve the speed of the entire calculation process of the operation data amount. And the number of index numbers in the second index of the filtering condition is the same as the number of hash functions used for conversion. When multiple hash functions are used to convert the filtering condition, the hash functions are used to calculate the filtering condition respectively to obtain the binary calculation result, and then each binary calculation result is converted into a decimal value, and each decimal value is taken as an index number in the second index of the filtering condition.

[0167] When the number of index numbers in the second index of the filtering condition is one, the number of times that the element in the bit array indicated by the second index is set to 1 is calculated, and the calculation result is taken as the operation data amount of the source table, specifically: the number of times that the corresponding element of the second index in the bit array of the source table is set to 1 is directly taken as the operation data amount of the source table.

[0168] When the number of index numbers in the second index of the filtering condition is multiple, the number of times that the element in the bit array indicated by the second index is set to 1 is calculated, and the calculation result is taken as the operation data amount of the source table, specifically: the minimum value is selected from the multiple times corresponding to the multiple index numbers as the operation data amount of the source table, or the maximum value is selected from the multiple times corresponding to the multiple index numbers as the operation data amount of the source table, or the average value of the multiple times corresponding to the multiple index numbers is calculated as the operation data amount of the source table. In this way, multiple ways of calculating the operation data amount can be provided to flexibly adapt to different calculation environments and calculation requirements, and the universality of the technical scheme is also embodied.

[0169] In order to facilitate understanding, the specific application of the above-mentioned calculation method related to the filtering condition can be referred to the following (I) Example 1 for the specific process of applying step S103 of calculating the operation data amount of the source table according to the bit array of the source table.

[0170] When the screening condition is a connection condition, the operation data volume of the source table is calculated according to the bit array of the source table, specifically: the second column set associated data is screened in the association table according to the connection condition, the second column set associated data is converted into the second index one by one, and the number of times that the elements in the bit array respectively indicated by the second index are set to 1 is calculated, and the calculation result is taken as the operation data volume of the source table.

[0171] The hash function used for converting the second column set associated data into the second index is the same as the hash function used for converting the first column set associated data into the first index in the above-mentioned step S102, so the simple conversion of the second column set associated data can also be realized, so as to quickly obtain the second index, and further improve the speed of the entire calculation process of the operation data volume. And the number of index numbers in the second index of one second column set associated data is the same as the number of hash functions used for conversion. When multiple hash functions are used to convert one second column set associated data, the hash functions are used to calculate the second column set associated data respectively to obtain binary calculation results, and then each binary calculation result is converted into a decimal value, and each decimal value is taken as an index number in the second index of the second column set associated data.

[0172] In the case where the number of index numbers in the second index of one second column set associated data is one, the above-mentioned calculation of the number of times that the elements in the bit array respectively indicated by the second index are set to 1, and the calculation result is taken as the operation data volume of the source table, specifically: the sum of the times corresponding to all second indexes of the second column set associated data is taken as the operation data volume of the source table.

[0173] In the case where the number of index numbers in the second index of one second column set associated data is multiple, the above-mentioned calculation of the number of times that the elements in the bit array respectively indicated by the second index are set to 1, and the calculation result is taken as the operation data volume of the source table, specifically: the minimum value is selected from the multiple times corresponding to the second index of each second column set associated data, and the sum of all minimum values is taken as the operation data volume of the source table; or the maximum value is selected from the multiple times corresponding to the second index of each second column set associated data, and the sum of all maximum values is taken as the operation data volume of the source table; or the average value of the multiple times corresponding to the second index of each second column set associated data is calculated, and the sum of all average values is taken as the operation data volume of the source table. In this way, multiple ways of calculating the operation data volume can be provided, which can flexibly adapt to different calculation environments and calculation requirements, and also embodies the universality of the technical scheme.

[0174] Optionally, the minimum value is selected from the multiple times corresponding to the second index of the data associated with each second column set, and the sum of all the minimum values is taken as the operation data amount of the source table. The specific numerical value of the operation data amount can be greatly reduced while ensuring the calculation accuracy of the operation data amount.

[0175] For the convenience of understanding, the specific application of the calculation method related to the connection condition can refer to the following (II) Example 2.

[0176] It should be understood that the calculation process of the operation data amount of the source table when the above-mentioned filtering conditions and connection conditions are respectively taken as the filtering conditions and connection conditions is only an example, which is not limited here.

[0177] S104: A generating unit in the database management system generates a logical plan according to the operation data amount of the source table.

[0178] The generating unit can be the generating unit 223 in the logical unit 220 in the database management system 200 in FIG. 1.

[0179] In some possible implementation manners, the process in which the database management system generates a logical plan according to the operation data amount of the source table can refer to the following implementation manner 1 and implementation manner 2:

[0180] Implementation manner 1: The database management system directly generates a logical plan according to the operation data amount of the source table.

[0181] Implementation manner 2: The database management system first generates an initial logical plan, and then optimizes the initial logical plan according to the operation data amount of the source table to obtain an optimized logical plan.

[0182] In the implementation manner 1 of step S104, the database management system directly generates a logical plan according to the operation data amount of the source table, specifically: according to the operation data amount of the source table, the connection algorithm, index method or connection order of the source table is selected, and then the determined connection algorithm, index method or connection order of the source table is taken as an important part of the logical plan to obtain the logical plan. This process can refer to the process in which a logical planner such as a parser in MySQL, a query rewriter in PostgreSQL or a logical planner in SQLite converts a user request into a logical plan by taking the statistical information of the source table as a configuration parameter. For the sake of brevity of the description, the process will not be described here.

[0183] In implementation manner 2 of step S104, the database management system first generates an initial logical plan, and then optimizes the initial logical plan according to the operation data volume of the source table to obtain an optimized logical plan. Specifically, whether the information (such as a connection algorithm, an index mode, or a connection order, etc.) of the source table in the initial logical plan is appropriate is determined according to the operation data volume of the source table, in a case where it is determined that the information of the source table in the initial logical plan is inappropriate, the connection algorithm, the index mode, or the connection order, etc. of the source table is adjusted according to the operation data volume of the source table, and then the plurality of adjustment results are taken as important components of a plurality of candidate logical plans, respectively, and finally one of the plurality of candidate logical plans is selected as the optimized logical plan. This process can refer to a process in which a query optimizer (such as a rule-based optimizer (RBO), a statistics-based optimizer, a dynamic optimizer, or an adaptive optimizer, etc.) in a database management system generates a plurality of candidate logical plans according to statistical information of a source table, and selects one of the plurality of candidate logical plans as a best logical plan. For the sake of brevity of the description, details are not described herein.

[0184] It should be understood that the implementation manner 1 and the implementation manner 2 of generating the logical plan in the above step S104 are only taken as an example, and are not limited specifically herein. In actual applications, implementation manners of generating a logical plan according to an operation data volume of a source table are all within the protection scope of the present application.

[0185] S105: A physical unit in the database management system generates a physical plan according to the logical plan.

[0186] The physical unit can be the physical unit 230 in the database management system 200 in FIG. 1.

[0187] The physical plan is an actual execution plan of the logical plan under a specific database system configuration and data distribution. The physical plan describes a process of executing the database operation in the logical plan at a physical level.

[0188] In some possible implementation manners, the database management system generating the physical plan corresponding to the logical plan generally includes selecting a suitable access path (such as a full table scan, an index scan, and the like), determining an execution order of operations, selecting a connection algorithm, and the like. The process can refer to a process of converting the logical plan into the physical plan by using a physical planner such as an optimizer-driven physical planner or a rule-based physical planner, and for the sake of brevity of the description, details are not described herein.

[0189] S106: An execution unit in the database management system executes the physical plan to obtain the request result.

[0190] The execution unit can be the execution unit 240 in the database management system 200 in FIG. 1.

[0191] In some possible implementation manners, the database management system executes the physical plan to obtain the request result, including: converting the physical plan into access instructions, sending the access instructions to the computing node, and receiving the request result obtained by the computing node according to the access instructions. The computing node is configured to process data in the data node according to the access instructions to obtain the request result. The computing node can be the computing node 300 in the database system in FIG. 1, and the data node can be the data node 400 in the database system in FIG. 1. The process of converting the physical plan into the access instructions can refer to a process of converting the physical plan into the access instructions by using a code generator such as Apache Calcite, Apache Phoenix, Presto SQL, or Apache Drill, and for the sake of brevity of the description, details are not described herein.

[0192] S107: The database management system sends the request result to the client.

[0193] Correspondingly, the client receives the request result from the database management system.

[0194] In some possible implementation manners, the database management system dynamically optimizes the logical plan, so that the specific content of the logical plan can be dynamically adjusted according to the actual execution of the logical plan (that is, the execution progress of the physical plan), thereby improving the query performance and efficiency.

[0195] The dynamic optimization manner of the logical plan at least includes the following three implementation manners:

[0196] Implementation 1: The database management system dynamically optimizes the logical plan according to the update of the statistics information of the source table.

[0197] Implementation 2: The database management system dynamically optimizes the logical plan according to the dynamic change of the source table.

[0198] Implementation 3: The database management system dynamically optimizes the logical plan according to the update of the statistics information of the source table and the dynamic change of the source table.

[0199] In the above-mentioned implementation 1, the database management system dynamically optimizes the logical plan according to the update of the statistics information of the source table, including: the database management system obtains a new logical plan based on the updated statistics information of the source table after each update of the statistics information of the source table, so that the logical plan is updated along with the update of the statistics information of the source table.

[0200] In the above-mentioned implementation 1, the database management system dynamically optimizes the logical plan according to the update of the statistics information of the source table, including: the database management system obtains a new logical plan based on the updated statistics information of the source table after each update of the statistics information of the source table, so that the logical plan is updated along with the update of the statistics information of the source table.

[0201] Compared with updating the statistics information of all tables in all data nodes immediately after performing a modification operation on any one table, and optimizing the logical plan according to the updated statistics information of the source table, the implementation 1 can greatly reduce the system resources consumed by the database management system for updating the statistics information of the source table.

[0202] In the above-mentioned implementation 2, the database management system dynamically optimizes the logical plan according to the dynamic change of the source table, which is embodied in the following two mechanisms:

[0203] In the first mechanism, the dynamic change of the source table depends on the change of the steps. Specifically, for steps independent of each other in the logical plan, the source table of one step and the source table of another step can be different. For example, in the logical plan L2 in the foregoing two, the source table of step 1 is table orders and table customers, and the source table of step 5 is table orders and table products.

[0204] Therefore, the first mechanism is specifically as follows: after the database management system calculates the statistics information, the bit array, and the operation data volume of the source table of a step, and finally obtains a logical plan according to the operation data volume of the source table of the step, the database management system calculates the statistics information, the bit array, and the operation data volume of the source table of another step, and then obtains a new logical plan according to the operation data volume of the source table of the another step, so as to realize dynamic optimization of the logical plan.

[0205] In the second mechanism, the dynamic change of the source table is embodied as dynamic generation of the source table. Specifically, for steps with a dependency relationship in the logical plan, the target table of a step can be the source table of another step, which often occurs in complex queries, multi-layer queries, and other query operations. The target table is a table on which an operation is completed in a step of the logical plan. For example, in the logical plan L2 in the foregoing second example, the source table of step 7 is the table formed by the retained row data of step 6, that is, the target table of step 6; and the source table of step 8 is the target table of step 7.

[0206] Therefore, the second mechanism is specifically as follows: after the database management system calculates the statistics information, the bit array, and the operation data volume of the source table of a step, and finally obtains a logical plan according to the operation data volume of the source table of the step, the database management system needs to wait until the target table of the step is obtained after the step in the logical plan is executed, and then takes the target table of the step as the source table of another step, and then calculates the statistics information, the bit array, and the operation data volume of the source table of the another step, and then obtains a new logical plan according to the operation data volume of the source table of the another step, so as to realize dynamic optimization of the logical plan.

[0207] In the above-mentioned implementation manner 3, the database management system dynamically optimizes the logical plan according to the update of the statistics information of the source table and the dynamic change of the source table, including: the database management system dynamically optimizes the logical plan according to the dynamic change of the source table, and constantly generates a new logical plan. Moreover, in the case where the cumulative value of the data change volume is greater than or equal to the statistical threshold, the database management system optimizes the newly generated logical plan again according to the updated statistics information of the source table, and obtains the latest logical plan.

[0208] The process in which the database management system dynamically optimizes the logical plan according to the dynamic change of the source table and constantly generates a new logical plan can refer to the above-mentioned implementation manner 2, and the process in which the database management system optimizes the new logical plan again according to the updated statistics information of the source table and obtains the latest logical plan can refer to the above-mentioned implementation manner 1. For the sake of brevity of the description, the details are not described herein.

[0209] In summary, by implementing the present application, the following advantages are achieved:

[0210] Firstly, since the statistical information of the source table describes the capacity or data volume of the source table, and the operation data volume of the source table describes the data volume of the source table that meets the filtering condition, the operation data volume of the source table is much smaller than the statistical information of the source table, and the operation data volume of the source table can more accurately reflect the actual system resource demand of executing the database operation. Therefore, compared with directly generating a logical plan based on the statistical information of the source table, the logical plan prepared based on the operation data volume in the present technical solution is more reasonable and can better reflect the actual demand, which can improve the rationality of the physical plan corresponding to the logical plan, and equivalently improve the execution efficiency of the user request.

[0211] Secondly, in the present technical solution, the first column set associated data related to the filtering condition in the source table is first mapped into a bit array, and then the mapping result of the first column set associated data in the bit array is used to calculate the operation data volume. In this way, not only the storage of the first column set associated data can be realized, but also the application of the stored first column set associated data can be realized. Compared with directly storing the first column set associated data, since one bit in the bit array can represent the presence or absence of one data, the storage space can be greatly reduced by using the bit array to store the first column set associated data. In addition, the operation of the bit array is at the bit level, supports bit operation, and supports parallel computing. Therefore, when the amount of first column set associated data is large, the subsequent calculation of the operation data volume using the mapping result of a large amount of first column set associated data in the bit array can take advantage of parallel computing to improve the calculation efficiency.

[0212] Thirdly, in the present technical solution, the operation data volume of the source table is calculated based on the mapping result of the data in the source table in the bit array. Compared with the way of actually executing the database operation on the source table to obtain the operation data volume of the source table, the mapping method in the present technical solution can quickly calculate the operation data volume of the source table, avoiding the indirect negative impact of long operation data volume acquisition time on the execution efficiency of the user request. The following two examples are used for further illustration.

[0213] As an example, in the case where the filtering condition is a filtering condition, the data related to the filtering condition in the source table is mapped into a bit array, the filtering condition is also mapped into the same bit array, and then the operation data volume of the source table is determined according to the relationship between the mapping result of the data related to the filtering condition in the source table and the mapping result of the filtering condition. In this way, the operation data volume can be quickly obtained, and the calculated operation data volume is accurate.

[0214] As another example, in the case where the screening condition is a connection condition, the operation data amount of the source table is determined according to the relationship between the mapping result of the data related to the filtering condition in the source table and the mapping result of the data related to the filtering condition in the association table by mapping the data related to the connection condition in the source table into a bit array and mapping the data related to the connection condition in the association table into the same bit array, so that the operation data amount can be quickly obtained, and the calculated operation data amount is accurate.

[0215] From the foregoing two, the logical plan L1 and the logical plan L2 in the logical plan, it can be known that one step in the logical plan only operates on a single table or a double table, that is, the number of source tables in one step is one or two. Next, taking the number of source tables as one and the number of source tables as two as examples, and taking the screening condition as a filtering condition when the number of source tables is one and taking the screening condition as a connection condition when the number of source tables is two, the execution process of the generation method of the logical plan of FIG. 2 will be specifically introduced, so as to illustrate the universality of the generation method of the logical plan of FIG. 2.

[0216] (I) Example 1: the number of source tables is one, and the screening condition is a filtering condition.

[0217] In the logical plan L1, the source table is the table employees, and the filtering condition of the source table is that the department name (dept_name) is "Sales".

[0218] The specific process of applying step S102 in the generation method of the logical plan of FIG. 2 is as follows:

[0219] First, the number of Boolean values in the bit array of the table employees is determined according to the statistical information of the table employees, and the bit array of the table employees is initialized.

[0220] Suppose the capacity of the table employees is 1 million bytes, and the capacity of the table employees is taken as the statistical information of the table employees.

[0221] The number of Boolean values in the bit array of the table employees is calculated by using the determination method 1 in step S102. The specific process is as follows: the decimal 1000000 (i.e. 1 million) is converted into binary to obtain 11110100001001000000 (i.e. a first value). The 11110100001001000000 is input into MD5, and the output result of MD5 is 1001 (i.e. a second value). The 1001 is converted into decimal to obtain 9 (i.e. a third value), and thus the number of Boolean values in the bit array of the table employees is 9. The result of the initialization of the bit array of the table employees can be seen from FIG. 3A, which is a schematic diagram of the initialization result of the bit array of a source table according to the present application.

[0222] Next, the data in the table employees is mapped into the bit array of the table employees, and the mapping result is taken as the bit array of the table employees. The process includes at least the following steps:

[0223] Step 1: the column data of the column with the column name "dept_name" in the table employees is filtered as a first column set according to the filter condition that the department name (dept_name) is "Sales", and each data in the first column set is taken as a first column set associated data.

[0224] Step 2: each department name (dept_name) in the table employees is assigned a binary value. For example, "Finance" = 00000, "Sales" = 00001, "Marketing" = 00010, and so on.

[0225] Step 3: the binary value corresponding to each first column set associated data is input into MD5 and SHA-1 respectively, the calculation results of the binary output by MD5 and SHA-1 are converted into decimal values, and the decimal values are taken as an index number in the first index of the first column set associated data. Thus, there are two index numbers in the first index of the first column set associated data. The following takes the first 5 first column set associated data shown in the table employees in the above example 1, user request as an example for description.

[0226] The first first column set associated data is "Finance", and thus 00000 is input into MD5 and SHA-1 respectively, and the calculation results of the binary output by MD5 and SHA-1 are 000 and 010. Thus, the decimal values 0 and 2 of 000 and 010 are taken as the index numbers in the first index of the first first column set associated data.

[0227] The first column set association data is "Sales", 00001 is input into MD5 and SHA-1 respectively, and the calculation results of the binary output of MD5 and SHA-1 are 011 and 111. Therefore, the decimal values 3 and 7 of 011 and 111 are taken as the index numbers in the first index of the second first column set association data.

[0228] The third first column set association data is the same as the second first column set association data, and the index numbers in the first index of the third first column set association data are also 3 and 7.

[0229] The fourth first column set association data is "Marketing", 00010 is input into MD5 and SHA-1 respectively, and the calculation results of the binary output of MD5 and SHA-1 are 100 and 110. Therefore, the decimal values 4 and 6 of 100 and 110 are taken as the index numbers in the first index of the fourth first column set association data.

[0230] The fifth first column set association data is the same as the fourth first column set association data, and the index numbers in the first index of the fifth first column set association data are also 4 and 6.

[0231] It should be understood that the process of calculating the first index of other first column set association data can refer to the process of calculating the first index of the first to fifth first column set association data in the table employees described above. For the sake of brevity of the description, the process will not be described again.

[0232] Step 4: The elements in the bit array of the table employees indicated by the first index of each first column set association data are set to 1 respectively, and the setting results obtained according to the first index of all first column set association data are taken as the bit array of the table employees. The bit array of the table employees can be seen from FIG. 3B, which is a schematic diagram of a bit array of a source table provided by the present application. For the sake of convenience of understanding, only the setting results obtained according to the first index of the first five first column set association data in the table employees are shown in FIG. 3B.

[0233] The specific case is as follows:

[0234] The index numbers in the first index of the first first column set association data are 0 and 2, and therefore the 0th bit and the 2nd bit in the bit array of the table employees are set to 1.

[0235] The index numbers in the first index of the second and third first column set association data are both 3 and 7, and therefore the 3rd bit and the 7th bit in the bit array of the table employees are set to 1 twice.

[0236] The index number in the first index of the fourth and fifth first column set associated data is 4 and 6 respectively, therefore, the fourth and sixth bit in the bit array of the table employees are set to 1 twice.

[0237] It should be understood that the process of setting the elements in the bit array of the table employees indicated by the first index of the other first column set associated data to 1 can refer to the process of setting the elements in the bit array of the table employees indicated by the first index of the first five first column set associated data to 1, which will not be expanded here for the sake of brevity of the description.

[0238] The specific process of applying step S103 in the generation method of the logical plan of FIG. 2 is as follows:

[0239] First, the filter condition of the table employees, i.e., the department name is "Sales", is converted into the second index. Specifically, the binary value 00001 corresponding to the department name "Sales" is input into MD5 and SHA-1 respectively, and the calculation results of the binary output of MD5 and SHA-1 are 011 and 111 respectively, therefore, the index numbers 3 and 7 in the second index of the filter condition are obtained.

[0240] Next, the number of times that the third and seventh bits in the bit array of the table employees are set to 1 is obtained. The average of the number of times that the third bit is set to 1 and the number of times that the seventh bit is set to 1 is calculated, and the average is taken as the operation data volume of the table employees.

[0241] The specific process of applying step S104 in the generation method of the logical plan of FIG. 2 is as follows: according to the operation data volume of the table employees, the index mode of the table employees is selected, and the determined index mode of the table employees is taken as an important part of the logical plan to obtain the logical plan L1'.

[0242] In summary, it can be known that if there are only the first 5 column sets of associated data in the table employees, since the 3rd bit and the 7th bit in the bit array of the table employees obtained according to the first index of the first 5 column sets of associated data are set to 1 twice (as shown in FIG. 3B), the operation data amount of the table employees is 2 column sets of associated data. Compared with the statistical information of the table employees in the filtering operation (i.e., the capacity of the table employees - 1 million bytes or the data amount - 5 column sets of associated data), the operation data amount of the table employees calculated by the foregoing generation method of the logical plan of FIG. 2 is smaller, more accurate, and more in line with the actual system resource demand when the filtering operation is performed. It can be understood that even if there are more column sets of associated data in the table employees, the operation data amount of the table employees calculated by the foregoing generation method of the logical plan of FIG. 2 is still much smaller than the statistical information of the table employees in the filtering operation.

[0243] Therefore, the logical plan obtained based on the operation data amount accurately reflecting the actual system resource demand when the database operation is performed can make the query more efficient and reduce the query cost. For example, based on the operation data amount of the table employees (specifically, 2), the logical plan will select the use of the index to perform the step 2-filter in the foregoing two, the logical plan L1 of the logical plan. If based on the statistical information of the table employees in the filtering operation (specifically, 1 million bytes), the logical plan will select the use of the full table scan to perform the step 2-filter in the foregoing two, the logical plan L1 of the logical plan. Compared with the full table scan, the use of the index can directly skip most of the data in the table employees and locate the data meeting the filtering condition in a shorter time, thereby reducing the query time, reducing the number of I / O operations, and reducing the system resources (such as CPU, memory, etc.) consumed in the query process.

[0244] (II) Example 2: The number of source tables is multiple, and the filtering condition is a connection condition.

[0245] The logical plan L2 involves the table orders, the table customers, and the table products. In the step 3-connection of the logical plan L2, the table customers can be taken as a source table, and the table orders can be taken as an associated table. The connection condition of the source table is the same customer number (customer_id).

[0246] The step S102 in the foregoing generation method of the logical plan of FIG. 2 is applied, and the specific process is as follows:

[0247] First, the number of Booleans in the bit array of table customers is determined according to the statistical information of table customers, and the bit array of table customers is initialized.

[0248] Suppose the data amount of table customers is 50 rows, and the data amount of table customers is taken as the statistical information of table customers.

[0249] The number of Booleans in the bit array of table customers is calculated by using the determination method 1 in step S102, and the specific process is as follows: the decimal 50 is converted into binary to obtain 110010 (i.e., the first value). The 110010 is input into MD5 to obtain the output result of MD5 as 1101 (i.e., the second value). The 1101 is converted into decimal to obtain 13 (i.e., the third value), and thus the number of Booleans in the bit array of table customers is 13. The result of the initialization of the bit array of table customers can be seen from FIG. 4A, which is a schematic diagram of the initialization result of the bit array of another source table provided by the present application.

[0250] Next, the data in table customers is mapped into the bit array of table customers, and the mapping result is taken as the bit array of table customers. The process at least includes the following steps:

[0251] Step 1: according to the connection condition that the same customer ID (customer_id) in table customers, the column data with the column name of "customer ID (customer_id)" is filtered as a first column set, and each data in the first column set is taken as a first column set association data.

[0252] Step 2: each customer ID (customer_id) in table customers is assigned a binary value. For example, "C0001" = 000000, "C0002" = 000001, "C0003" = 000010, "C0004" = 000011, "C0005" = 000100, and so on.

[0253] Step 3: input the binary value corresponding to each first column set associated data pair into MD5, SHA-1, SHA-256 respectively, convert the binary calculation results output by MD5, SHA-1, SHA-256 into decimal values, and take the decimal values as one index number in the first index of the first column set associated data, and there are three index numbers in the first index of the first column set associated data. The following takes the first five first column set associated data shown in the customers table in the above-mentioned first one, user request, as an example for illustration.

[0254] The first first column set associated data is "C0001", so input 000000 into MD5, SHA-1, SHA-256 respectively, and get the binary calculation results output by MD5, SHA-1, SHA-256 as 0001, 0010, 0100, so take the decimal values 1, 2, 4 of 0001, 0010, 0100 as the index number in the first index of the first first column set associated data.

[0255] The second first column set associated data is "C0002", so input 000001 into MD5, SHA-1, SHA-256 respectively, and get the binary calculation results output by MD5, SHA-1, SHA-256 as 0100, 1000, 1011, so take the decimal values 4, 8, 11 of 0100, 1000, 1011 as the index number in the first index of the second first column set associated data.

[0256] The third first column set associated data is "C0003", so input 000010 into MD5, SHA-1, SHA-256 respectively, and get the binary calculation results output by MD5, SHA-1, SHA-256 as 0011, 0100, 0101, so take the decimal values 3, 4, 5 of 0011, 0100, 0101 as the index number in the first index of the third first column set associated data.

[0257] The fourth first column set associated data is "C0004", so input 000011 into MD5, SHA-1, SHA-256 respectively, and get the binary calculation results output by MD5, SHA-1, SHA-256 as 0100, 0110, 0011, so take the decimal values 4, 6, 3 of 0100, 0110, 0011 as the index number in the first index of the fourth first column set associated data.

[0258] The first index of the fifth first column set associated data is C0005, and 000100 is input into MD5, SHA-1 and SHA-256, respectively, to obtain the binary calculation results of the MD5, SHA-1 and SHA-256 outputs as 1001, 0011 and 0111. Therefore, the decimal values 9, 3 and 7 of 1001, 0011 and 0111 are taken as index numbers in the first index of the fifth first column set associated data.

[0259] It should be understood that the process of calculating the first index of the other 45 first column set associated data can refer to the process of calculating the first index of the first first column set associated data to the fifth first column set associated data in the table customers described above. For the sake of brevity of the description, the process will not be described in detail here.

[0260] Step 4: Set the elements in the bit array of the table customers indicated by the first index of each first column set associated data to 1, respectively, and set the setting results obtained according to the first index of all the first column set associated data as the bit array of the table customers. The bit array of the table customers can be seen from FIG. 4B, which is a schematic diagram of a bit array of another source table provided by the present application. For the sake of convenience of understanding, only the setting results obtained according to the first index of the first five first column set associated data in the table customers are shown in FIG. 4B. The specific cases are as follows:

[0261] The index numbers in the first index of the first first column set associated data are 1, 2 and 4, and therefore the first, second and fourth bits in the bit array of the table employees are set to 1.

[0262] The index numbers in the first index of the second first column set associated data are 4, 8 and 11, and therefore the fourth, eighth and eleventh bits in the bit array of the table employees are set to 1.

[0263] The index numbers in the first index of the third first column set associated data are 3, 4 and 5, and therefore the third, fourth and fifth bits in the bit array of the table employees are set to 1.

[0264] The index numbers in the first index of the fourth first column set associated data are 4, 6 and 3, and therefore the fourth, sixth and third bits in the bit array of the table employees are set to 1.

[0265] The index numbers in the first index of the fifth first column set associated data are 9, 3 and 7, and therefore the ninth, third and seventh bits in the bit array of the table employees are set to 1.

[0266] It should be understood that the process of setting the elements in the bit array of the table customers indicated by the first index of the other 45 first column set associated data to 1 can refer to the process of setting the elements in the bit array of the table customers indicated by the first index of the first 5 first column set associated data to 1, which will not be expanded here for the sake of brevity of the description.

[0267] The specific process of applying step S103 in the generation method of the logical plan of FIG. 2 is as follows:

[0268] First, according to the connection condition that the same customer ID (customer_id) in the associated table - table orders, the column data of the column with the column name "customer ID (customer_id)" is filtered as a second column set, and each data in the second column set is taken as a second column set associated data.

[0269] Next, each second column set associated data is converted into a second index. Specifically, the binary value corresponding to each second column set associated data is input into MD5, SHA-1, and SHA-256, respectively, and the calculation results of the binary output by MD5, SHA-1, and SHA-256 are all converted into decimal numerical values, and the decimal numerical values are all taken as an index number in the second index of the second column set associated data. Then, there are three index numbers in the second index of the second column set associated data.

[0270] Then, for the second index of each second column set associated data, the number of times the elements in the bit array of the table customers indicated by the three index numbers in the second index are set to 1 is obtained, and the minimum value is selected from the three obtained times.

[0271] Finally, all the minimum values are added to obtain the sum of all the minimum values as the operation data volume of the table customers.

[0272] Next, the first 5 second column set associated data shown in the table orders in the aforementioned first user request will be taken as an example for description, and it is assumed that the number of times each element in the bit array of the table customers is set to 1 is shown in Table 7 as follows.

[0273] Table 7

[0274] The binary values 000000 corresponding to the second column set association data "C0001" are input into MD5, SHA-1, and SHA-256, respectively, and the decimal values 1, 2, and 4 of the calculation results 0001, 0010, and 0100 of the binary output by MD5, SHA-1, and SHA-256 are used as the index numbers in the second index of the first second column set association data. The number of times that the first, second, and fourth bits of the bit array of the table employees are set to 1 is 17, 7, and 9, respectively, and the minimum value among 17, 7, and 9 is 7.

[0275] The binary values 000010 corresponding to the second column set association data "C0003" are input into MD5, SHA-1, and SHA-256, respectively, and the decimal values 3, 4, and 5 of the calculation results 0011, 0100, and 0101 of the binary output by MD5, SHA-1, and SHA-256 are used as the index numbers in the second index of the second second column set association data. The number of times that the third, fourth, and fifth bits of the bit array of the table employees are set to 1 is 20, 9, and 16, respectively, and the minimum value among 20, 9, and 16 is 9.

[0276] The third and fourth second column set association data are both "C0001", and thus the minimum value is the same as that of the first second column set association data, which is 7.

[0277] The binary values 000011 corresponding to the second column set association data "C0002" are input into MD5, SHA-1, and SHA-256, respectively, and the decimal values 4, 8, and 11 of the calculation results 0100, 1000, and 1011 of the binary output by MD5, SHA-1, and SHA-256 are used as the index numbers in the second index of the fifth second column set association data. The number of times that the fourth, eighth, and eleventh bits of the bit array of the table employees are set to 1 is 9, 12, and 14, respectively, and the minimum value among 9, 12, and 14 is 9.

[0278] The minimum values of the first to fifth second column set association data are added to obtain 49, and 49 is used as the operation data amount of the first five second column set association data of the table customers.

[0279] It should be understood that the process of converting other second column set associated data in the table orders to second indexes, obtaining minimum values according to the second indexes, and calculating the sum of all minimum values as the operation data amount of the table customers can refer to the process of obtaining the operation data amount of the first five second column set associated data in the table customers described above, and will not be described in detail here for the sake of brevity of the description.

[0280] According to the operation data amount of the table customers, the connection algorithm, the index mode, or the connection order of the table customers is selected, and the determined connection algorithm, the index mode, or the connection order of the table customers is taken as an important part of the logical plan to obtain the logical plan L2'.

[0281] As can be seen from the above, compared with the statistical information of the table customers in the connection operation (i.e., the product of the data amount of the table orders and the data amount of the table customers), the operation data amount of the table customers calculated by the logical plan generation method of FIG. 2 is less, more accurate, and more in line with the actual system resource demand when the connection operation is performed. For example, if there are only the first five second column set associated data in the table employees, the product of the data amount of the table orders and the data amount of the table customers is 5*50 = 250, and the statistical information of the table customers in the connection operation is 250. As can be seen from the above example, the operation data amount of the table customers calculated by the logical plan generation method of FIG. 2 is 49, which is less and can ensure the accuracy of the calculation and is in line with the actual system resource demand when the connection operation is performed. It can be understood that even if the table customers has more second column set associated data, the operation data amount of the table customers calculated by the logical plan generation method of FIG. 2 is much less than the statistical information of the table customers in the connection operation.

[0282] Therefore, the logical plan based on the operation data amount accurately reflecting the actual system resource demand when performing the database operation can make the system resource used by the query less. For example, the logical plan generated according to the operation data amount of 49 of the table customers can make the subsequent physical plan corresponding to the logical plan call one computing node to perform step 3 of the logical plan L2. If the logical plan is generated according to the statistical information of 250 of the table customers, it will make the subsequent physical plan corresponding to the logical plan call two computing nodes to perform step 3 of the logical plan L2. Therefore, compared with the statistical information with larger deviation, the operation data amount reflecting the actual system resource demand when performing the database operation less and more accurately can more reasonably utilize the system resource and reduce unnecessary system resource consumption.

[0283] It should be understood that for steps 5 and 6 of the logical plan L2, the calculation process of the operation data amount of the operation involving two tables, and the process of obtaining the logical plan based on the operation data amount, can refer to the calculation process of the operation data amount of the table customers in step 3 of the logical plan L2 and the process of obtaining the logical plan L2' based on the operation data amount of the table customers in the above (ii) Example 2 for brevity of the specification, which will not be expanded here.

[0284] The present application also provides a chip system, which comprises a processor and a power supply circuit, the power supply circuit is used to perform power supply for the processor, and the processor is used to perform the steps executed by the database management system in the above-mentioned generation method of the logical plan of FIG. 2. Specifically, the processor is used to perform the steps executed by the acquisition unit, the logical unit (including the mapping unit, the calculation unit, the generation unit), the physical unit and the execution unit in the database management system in the above-mentioned generation method of the logical plan of FIG. 2. For brevity, it will not be expanded here. Wherein, the processor can be realized by GPU, and also can be realized by DPU, NPU, XPU, SoC, offload card, acceleration card and other computing devices.

[0285] Optionally, when the acquisition unit, the logical unit (including the mapping unit, the calculation unit, the generation unit), the physical unit and the execution unit in the database management system in the above-mentioned generation method of the logical plan of FIG. 2 correspond one by one to the acquisition unit 210, the logical unit 220 (including the mapping unit 221, the calculation unit 222, the generation unit 223), the physical unit 230 and the execution unit 240 in the database management system 200 in the above-mentioned FIG. 1, the processor in the chip system can also be used to realize the function of the database management system 200 in the above-mentioned FIG. 1.

[0286] Referring to FIG. 5, FIG. 5 is a structural schematic diagram of a computing device provided by the present application. As shown in FIG. 5, the computing device 500 provided by the present application includes a bus 501, a processor 502, a memory 503, and a communication interface 504. The processor 502, the memory 503, and the communication interface 504 communicate through the bus 501. The computing device 500 can be a server or a terminal device. It should be understood that the number of processors and memories in the computing device 500 is not limited by the present application.

[0287] The bus 501 can be a peripheral component interconnect Express (PCIe) bus or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. Among them, the unified bus is also called a unified bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in FIG. 5, but it does not mean that there is only one bus or one type of bus. The bus 501 can include a path for transmitting information between various components (e.g., the memory 503, the processor 502, the communication interface 504) of the computing device 500.

[0288] The processor 502 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), an ASIC, an FPGA, a CPLD, an NPU, a SoC, an offload card, an acceleration card, etc. in a computing device.

[0289] The memory 503 can include volatile memory, such as random access memory (RAM). The processor 502 can also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). In addition, the memory 503 can also be implemented by storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0290] It is worth noting that the same type of storage medium can be configured to implement the function of the memory 503 in the same computing device, or two or more types of storage media can be configured to implement the function of the memory 503, which is not limited in the present application.

[0291] The memory 503 stores executable program code, and the processor 502 executes the executable program code to respectively implement the functions of the mapping unit 221, the calculation unit 222, and the generation unit 223 in the database management system 200 of FIG. 1, thereby respectively performing the steps performed by the mapping unit, the calculation unit, and the generation unit in the database management system in the generation method of the logical plan of FIG. 2.

[0292] Optionally, the program code in the memory 503 for implementing the functions of the mapping unit 221, the calculation unit 222, and the generation unit 223 can also be dynamically adjusted according to business needs, so that the program code stored in the memory 503 can also implement the functions of one or more of the acquisition unit 210, the logical unit 220 (including the mapping unit 221, the calculation unit 222, and the generation unit 223), the physical unit 230, and the execution unit 240 in the database management system 200 of FIG. 1. For example, the program code in the memory 503 for implementing the function of the mapping unit 221 can not only implement the function of the mapping unit 221 in the database management system 200 of FIG. 1, but also implement the functions of the acquisition unit 210, the physical unit 230, and the execution unit 240 in the database management system 200 of FIG. 1.

[0293] In summary, the memory 503 stores instructions for performing the method of generating the logical plan of FIG. 2.

[0294] The communication interface 504 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, etc., to enable communication between the computing device 500 and other computing devices or communication networks.

[0295] As a possible implementation, the computing device 500 can also include a chip system including a processor and a power supply circuit for performing power supply to the processor, and the processor is configured to perform the operation steps performed by the database management system in the method of generating the logical plan of FIG. 2. Alternatively, the processor can implement the functions of one or more of the obtaining unit 210, the logical unit 220 (including the mapping unit 221, the computing unit 222, and the generating unit 223), the physical unit 230, and the executing unit 240 in the database management system 200 of FIG. 1 by executing the program code stored in the memory 503 in the computing device 500, thereby implementing the operation steps performed by the database management system in the method of generating the logical plan of FIG. 2. The processor can be implemented by a GPU, or by a DPU, a NPU, an XPU, a SoC, an offload card, an acceleration card, or other computing devices or AI chips.

[0296] As a possible implementation, the computing device 500 can include multiple types of processors 502, i.e., the computing device 500 is a heterogeneous device, for example, the computing device 500 includes a CPU and a GPU, and at least one of the processors 502 can perform the operation steps performed by the database management system in the method of generating the logical plan of FIG. 2. For brevity, details are not repeated here.

[0297] Referring to FIG. 6, FIG. 6 is a structural schematic diagram of a computing device cluster provided by the present application. The computing device cluster provided by the present application includes at least one computing device. The computing device can be a server, for example, a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a desktop computer, a notebook computer, or a terminal device such as a smart phone.

[0298] As shown in FIG. 6, the computing device cluster includes at least one computing device 500. The memory 503 in one or more computing devices 500 in the computing device cluster can store the same instructions for performing the steps performed by the database management system in the method of generating the logical plan of FIG. 2.

[0299] In some possible implementation manners, the memory 503 of one or more of the computing devices 500 in the computing device cluster can also respectively store partial instructions for performing the steps performed by the database management system in the generation method of the logical plan of FIG. 2. In other words, the combination of the one or more computing devices 500 can collectively execute the instructions for performing the steps performed by the database management system in the generation method of the logical plan of FIG. 2.

[0300] It should be noted that the memory 503 in different computing devices 500 in the computing device cluster can store different instructions for respectively performing part of the functions of the database management system 200 in FIG. 1. That is, the instructions stored in the memory 503 in different computing devices 500 can implement the functions of one or more of the mapping unit 221, the computing unit 222, and the generation unit 223.

[0301] Optionally, the instructions in the memory 503 in different computing devices 500 for implementing the functions of one or more of the mapping unit 221, the computing unit 222, and the generation unit 223 can also be dynamically adjusted according to business requirements, such that the instructions stored in the memory 503 in different computing devices 500 can also implement the functions of one or more of the acquisition unit 210, the logical unit 220 (including the mapping unit 221, the computing unit 222, and the generation unit 223), the physical unit 230, and the execution unit 240 in the database management system 200 in FIG. 1.

[0302] Referring to FIG. 7, FIG. 7 is a structural schematic diagram of another computing device cluster provided in the present application. In some possible implementation manners, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc. As shown in FIG. 7, two computing devices 500A and 500B are connected through a network. Specifically, the computing devices are connected to the network through communication interfaces in the computing devices. In this type of possible implementation manner, the memory 503 in the computing device 500A stores instructions for performing the functions of the mapping unit 221. Meanwhile, the memory 503 in the computing device 500B stores instructions for performing the functions of the computing unit 222 and the generation unit 223.

[0303] The connection manner between the computing device cluster shown in FIG. 7 can be that the functions implemented by the mapping unit 221 are executed by the computing device 500A, considering that the generation method of the logical plan provided in the present application needs to receive a large number of user requests.

[0304] It should be understood that the functions of the computing device 500A shown in FIG. 7 can also be completed by multiple computing devices 500. Similarly, the functions of the computing device 500B can also be completed by multiple computing devices 500.

[0305] The present application also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection mode of the computing device cluster described with reference to FIG. 6 and FIG. 7. The difference is that the memory 503 in one or more computing devices 500 in the computing device cluster can store the same instructions for performing the steps performed by the database management system in the generation method of the logical plan of FIG. 2.

[0306] In some possible implementations, the memory 503 of one or more computing devices 500 in the computing device cluster can also respectively store partial instructions for performing the steps performed by the database management system in the generation method of the logical plan of FIG. 2. In other words, the combination of one or more computing devices 500 can collectively execute the instructions for performing the steps performed by the database management system in the generation method of the logical plan of FIG. 2.

[0307] It should be noted that the memory 503 in different computing devices 500 in the computing device cluster can store different instructions for respectively performing part of the functions of the database management system 200 in FIG. 1. That is, the instructions stored in the memory 503 in different computing devices 500 can implement the functions of one or more of the mapping unit 221, the computing unit 222, and the generation unit 223.

[0308] Optionally, the instructions in the memory 503 in different computing devices 500 for implementing the functions of one or more of the mapping unit 221, the computing unit 222, and the generation unit 223 can also be dynamically adjusted according to business needs, so that the instructions stored in the memory 503 in different computing devices 500 can also implement the functions of one or more of the acquisition unit 210, the logical unit 220 (including the mapping unit 221, the computing unit 222, and the generation unit 223), the physical unit 230, and the execution unit 240 of the database management system 200 in FIG. 1.

[0309] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0310] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a logical plan, the method comprising: The method comprises: obtaining a bit array of a source table, wherein the bit array comprises multi-bit Boolean values, and the bit array is used to indicate a mapping result between data in the source table and elements in the bit array; calculating an operation data amount of the source table according to the bit array, wherein the operation data amount is an amount of data in the source table that meets a filtering condition or a connection condition. generating a logical plan according to the operation data amount, wherein the logical plan is used to indicate execution of a database operation according to a user request.

2. The method of claim 1, wherein, Before the bit array of the source table is obtained, the method further comprises: filtering a first column set in the source table according to the filtering condition, and converting associated data of the first column set into a first index; setting elements in the bit array indicated by the first index to 1 respectively, to obtain the bit array.

3. The method of claim 2, wherein, The conversion of the associated data of the first column set into the first index comprises: calculating the associated data of the first column set by using a hash function to obtain binary calculation results, converting the binary calculation results into decimal numerical values, and taking the decimal numerical values as the first index.

4. The method according to any one of claims 1 to 3, characterized in that, In a case where the filtering condition is the filtering condition, the calculation of the operation data amount of the source table according to the bit array comprises: converting the filtering condition into a second index; calculating a number of times that elements in the bit array indicated by the second index are set to 1 to obtain the operation data amount.

5. The method of claim 4, wherein, The conversion of the filtering condition into the second index comprises: calculating the filtering condition by using a hash function to obtain binary calculation results, converting the binary calculation results into decimal numerical values, and taking the decimal numerical values as the second index.

6. The method according to claim 4 or 5, characterized in that, The calculation of the number of times that elements in the bit array indicated by the second index are set to 1 to obtain the operation data amount comprises: selecting a minimum value or a maximum value from the number of times that elements in the bit array indicated by the second index are set to 1 as the operation data amount, or calculating an average value of the number of times that elements in the bit array indicated by the second index are set to 1 as the operation data amount.

7. The method according to any one of claims 1 to 3, characterized in that, In a case where the filtering condition is the connection condition, the calculation of the operation data amount of the source table according to the bit array comprises: filtering a second column set in an associated table according to the connection condition, and converting associated data of the second column set into a second index, wherein the associated table is another table that is connected with the source table according to the connection condition; calculating a number of times that elements in the bit array indicated by the second index are set to 1 respectively to obtain the operation data amount.

8. The method of claim 7, wherein, The calculation of the number of times that elements in the bit array indicated by the second index are set to 1 respectively to obtain the operation data amount comprises: selecting a minimum value or a maximum value from the times that the elements in the bit array indicated by the second indexes are set to 1, or calculating an average value of the times that the elements in the bit array indicated by the second indexes are set to 1; taking a sum of the minimum values as the operation data amount, or taking a sum of the maximum values as the operation data amount, or taking a sum of the average values as the operation data amount.

9. A database system, characterized by comprising: a mapping unit, a calculating unit, and a generating unit, the mapping unit is configured to obtain a bit array of a source table, wherein the bit array comprises a plurality of Boolean values, and the bit array is used to indicate a mapping result between data in the source table and elements in the bit array; the calculating unit is configured to calculate an operation data amount of the source table according to the bit array, wherein the operation data amount is an amount of data in the source table that meets a filtering condition or a connection condition; the generating unit is configured to generate a logical plan according to the operation data amount, wherein the logical plan is used to indicate execution of a database operation according to a user request.

10. The system of claim 9, wherein, The system is further configured to implement the method of any one of claims 2 to 8.

11. A chip system, characterized by The chip system comprises a processor and a power supply circuit, the power supply circuit is configured to supply power to the processor, and the processor is configured to execute the operation steps of the method of any one of claims 1 to 8.

12. A computing device, comprising: comprising a processor and a memory, the memory is configured to store instructions, and the processor is configured to execute the instructions, when the processor executes the instructions, the method of any one of claims 1 to 8 is implemented.

13. A cluster of computing devices, characterized in that, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method of any one of claims 1 to 8.

14. A computer program product comprising instructions, characterized in that, When the instructions are executed by the computing device, the computing device executes the method of any one of claims 1 to 8.

15. A computer-readable storage medium, characterized in that, comprising computer program instructions, when the computer program instructions are executed by a computing device, the computing device executes the method of any one of claims 1 to 8.

Images