Database management systems, related apparatus, methods, and media

Abstract

The present disclosure provides a database management system, related devices, methods and media. The database management system comprises: a search space exploration unit configured to parse semantics and syntax of a current query statement according to a first scheduling request, and generate a plurality of first candidate query plans; a metadata management unit configured to acquire, at a predicted execution time, usage of a computing resource and first data from relevant data queried by the current query statement and arriving at a database according to a second scheduling request; a rule unit configured to expand, according to the first data, the plurality of first candidate query plans to generate second candidate query plans; and a query optimization unit configured to select, at the predicted execution time and in a case where the computing resource is sufficient, a target query plan from the plurality of first candidate query plans and the second candidate query plans based on execution cost, wherein the database is queried by executing the target query plan. The present disclosure improves database query efficiency and computing resource utilization.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and more particularly, to a database management system, related apparatus, method and medium. BACKGROUND

[0002] The development of database technology provides increasingly large data storage capabilities, and users can query and obtain required data from the mass data storage through a network or the like. With the increasing amount of data storage, the user's query demand for data in the database is becoming more and more complex, and often needs to continuously query the query results of specific events that meet certain range conditions (for example, time range). In the database continuous query scenario, the client can continuously send multiple query statements to the database management server, the database management server can generate a corresponding query plan according to each query statement, execute the query plan to perform query operation on the database, access the database to obtain relevant data and manipulate the relevant data to generate the query result corresponding to the query, and return the query result to the client. The query plan generated according to each query statement is usually fixed, while the relevant data queried by the query statement gradually arrives in the database over time and the computing resource usage of the database management server also changes over time, resulting in the situation that the database management server executes the query plan with low efficiency and / or insufficient computing power, which reduces the database query efficiency and the computing resource utilization of the database query. SUMMARY

[0003] In view of this, the present disclosure aims to improve the database query efficiency and computing resource utilization in the database continuous query scenario.

[0004] To achieve this purpose, according to one aspect of the present disclosure, a database management system is provided, comprising:

[0005] A search space exploration unit configured to parse semantics and syntax of a current query statement according to a first scheduling request, and generate a plurality of first candidate query plans;

[0006] A metadata management unit configured to, at a predicted execution time, according to a second scheduling request, obtain usage of computing resources and first data of relevant data queried by the current query statement and arrived in the database, the predicted execution time being a predicted execution time of each query step in a plurality of query steps of a query plan in the plurality of first candidate query plans;

[0007] A rule unit configured to, according to the first data, expand a second candidate query plan based on the plurality of first candidate query plans;

[0008] The query optimization unit is configured to select a target query plan from the plurality of first candidate query plans and the second candidate query plan based on execution cost at the predicted execution time and when computing resources are sufficient, and query the database by executing the target query plan.

[0009] Optionally, the database management system further comprises:

[0010] The execution unit is configured to execute the target query plan corresponding to the current query statement in an incremental query step manner based on a query result of a query step of the target query plan corresponding to a previous query statement, to generate a query result corresponding to the current query statement.

[0011] Optionally, the target query plan corresponding to the current query statement comprises a plurality of query steps, the plurality of query steps comprise a first query step and a second query step, the first query step is a query step that has been executed in the target query plan corresponding to the previous query statement, and the second query step is a query step that has not been executed in the target query plan corresponding to the previous query statement. The database management system further comprises:

[0012] The storage unit is configured to store a first query result obtained by executing the first query step in the target query plan corresponding to the previous query statement.

[0013] Optionally, the query optimization unit comprises a first reading module, an execution cost estimation module, and a query plan selection module,

[0014] The first reading module is configured to read the first query result when the first query step needs to be executed.

[0015] The execution cost estimation module is configured to estimate execution cost of the plurality of first candidate query plans and the second candidate query plan by using an execution cost estimation method based on the first query result, wherein the first query result is read and the first query step is skipped during execution cost estimation.

[0016] The query plan selection module is configured to select a query plan with minimum execution cost from the plurality of first candidate query plans and the second candidate query plan as the target query plan.

[0017] Optionally, the first candidate query plan comprises a plurality of query steps, and the rule unit comprises a query step selection module and a query plan expansion module,

[0018] The query step selection module is configured to select a third query step that takes the first data as input data from the first candidate query plan.

[0019] The query plan expansion module is configured to adjust an operator and an operator execution order of the third query step to generate the second candidate query plan.

[0020] Optionally, the execution unit comprises a second reading module, a query plan execution module and a query result generation module.

[0021] The second reading module is configured to read the first query result when the first query step needs to be executed.

[0022] The query plan execution module is configured to execute the plurality of query steps according to a target query plan corresponding to the current query statement, wherein the second query result is obtained by executing the second query step, and the first query step is skipped by reading the first query result.

[0023] The query result generation module is configured to generate a query result corresponding to the current query statement based on the first query result and the second query result.

[0024] Optionally, the database management system further comprises:

[0025] A job management unit configured to register a database query job corresponding to the current query statement.

[0026] A scheduling unit configured to send the first scheduling request after the database query job corresponding to the current query statement is successfully registered.

[0027] Optionally, the predicted execution time comprises a plurality of predicted execution times, and the scheduling unit is further configured to preset the plurality of predicted execution times, and send the second scheduling request when the predicted execution time and the computing resource are sufficient.

[0028] According to an aspect of the present disclosure, a database management engine is provided, comprising:

[0029] A search space exploration unit configured to parse semantics and syntax of a current query statement according to a first scheduling request, and generate a plurality of first candidate query plans.

[0030] A metadata management unit configured to obtain, according to a second scheduling request, a usage of a computing resource and first data reaching a database in related data queried by the current query statement at a predicted execution time of each query step in a plurality of query steps of one query plan in the plurality of first candidate query plans.

[0031] A rule unit configured to generate, according to the first data, a second candidate query plan based on the plurality of first candidate query plans.

[0032] A query optimization unit is configured to select a target query plan from the plurality of first candidate query plans and the second candidate query plan based on execution cost at the predicted execution time and when computing resources are sufficient, wherein the database is queried by executing the target query plan.

[0033] According to an aspect of the present disclosure, there is provided a computing device comprising:

[0034] a memory configured to store computer executable code;

[0035] a processor configured to execute the computer executable code; and

[0036] the database management system of any of the above.

[0037] According to an aspect of the present disclosure, there is provided a system on chip comprising:

[0038] a memory configured to store computer executable code;

[0039] a processor configured to execute the computer executable code; and

[0040] the database management system of any of the above.

[0041] According to an aspect of the present disclosure, there is provided a database management method comprising:

[0042] parsing semantics and syntax of the current query statement according to a first scheduling request to generate a plurality of first candidate query plans;

[0043] obtaining usage of computing resources and first data reaching the database from related data queried by the current query statement according to a second scheduling request at a predicted execution time, the predicted execution time being a predicted execution time of each query step in a query plan of the plurality of first candidate query plans;

[0044] generating a second candidate query plan based on the plurality of first candidate query plans according to the first data;

[0045] selecting a target query plan from the plurality of first candidate query plans and the second candidate query plan based on execution cost at the predicted execution time and when computing resources are sufficient, wherein the database is queried by executing the target query plan.

[0046] According to an aspect of the present disclosure, there is provided a computer readable medium comprising computer executable code which, when executed by a processor, implements the method of the above.

[0047] In the embodiments of the present disclosure, the characteristics that the relevant data queried by the current query statement reaches the database in time sequence and the computing resource usage of the database management system changes over time in the continuous query database scenario are utilized. The semantics and syntax of the current query statement are parsed according to the first scheduling request to generate a plurality of first candidate query plans. The usage of the computing resource and the first data of the relevant data queried by the current query statement reaching the database are obtained according to the second scheduling request at the predicted execution time of each query step in a plurality of query steps of a first candidate query plan (which can be an optimal query plan in the plurality of first candidate query plans). Then, the second candidate query plan is generated based on the first data and the plurality of first candidate query plans. In the case that the computing resource is sufficient at the predicted execution time, the target query plan is selected from the plurality of first candidate query plans and the second candidate query plan based on the execution cost. In this way, the target query plan at the current time is found at each predicted execution time in the process of executing the current query statement, which reduces the execution cost of the entire query process of executing the current query statement, thereby improving the database query efficiency. The target query plan at the current time found at the predicted execution time is executed in the case that the current computing resource of the database management server is sufficient, which ensures the normal execution of the current query statement, and improves the utilization of the computing resource. BRIEF DESCRIPTION OF DRAWINGS

[0048] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0049] Figure 1 An internal structure diagram of a database access system to which the embodiments of the present disclosure are applied is shown;

[0050] Figure 2 An internal structure diagram of a database management server according to the embodiments of the present disclosure is shown;

[0051] Figure 3 An internal structure diagram of a database management system according to the embodiments of the present disclosure is shown;

[0052] Figure 4 A schematic diagram of a first candidate query plan and a second candidate query plan according to the embodiments of the present disclosure is shown;

[0053] Figure 5 A schematic diagram of executing the target query plan at a plurality of predicted execution times according to the embodiments of the present disclosure is shown;

[0054] Figure 6 A flowchart of a database management method according to the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0055] The present disclosure is described in detail below based on examples, but the present disclosure is not limited to only these examples. In the following detailed description of the present disclosure, some specific details are described in detail. The present disclosure can also be fully understood without the description of these details. In order to avoid confusion of the essence of the present disclosure, well-known methods, processes, and procedures are not described in detail. In addition, the drawings are not necessarily drawn to scale.

[0056] The following terms are used herein.

[0057] Database: is a warehouse that organizes, stores and manages data according to a certain data structure, which can be a relational database. A relational database is a database created on the basis of a relational model, which is a two-dimensional table model for organizing data into multiple tables composed of rows and columns. As the amount of data in the database becomes larger and larger, database partitioning emerges. Corresponding to database partitioning, the data in the database partition is organized into multiple partition data tables. A tuple is a data record in a data table. The tuples of the partition data table are clustered on the storage device in a physical partitioning strategy. The partition granularity is the unit of partitioning, for example, partitioning by month or year.

[0058] Query statement: is a database query and programming language for accessing data and querying, updating and managing databases. In addition to querying data in the database, a query statement also needs to operate on the data set read, so a query statement can include one or more operators, and different query statements contain different operators. The query statement is, for example, an SQL (Structured Query Language, structured query language) statement. SQL refers to a special programming language specifically for managing data stored in a relational database. SQL can refer to various types of data-related languages, including data definition languages and data manipulation languages, where the scope of SQL can include data insertion, query, update and deletion, schema creation and modification, and data access control.

[0059] Query plan: also known as execution plan, the query plan is a collection of logical operations that the execution unit actually executes the query statement.

[0060] Optimal query plan: can be a query plan with the smallest execution cost among multiple candidate query plans. The execution cost of a query plan is related to the characteristics of the operator and the execution order of the operator.

[0061] Incremental query planning (TIP) model based on time-varying relation (TVR): an algebraic definition proposed on the basis of time-varying relation, for example, "snapshot R_t" represents the instance of R at time point t, and "ΔR_{t1}^{t2}" represents the change of R from time point t1 to time point t2. On this basis, a series of basic operations can be defined, for example, "R_{t1}+ΔR_{t1}^{t2}=R_{t2}", in different cases, "+" may mean the simple combination of R_{t1} and ΔR_{t1}^{t2}, or a more complex aggregation operation.

[0062] Computing cluster: a computer cluster composed of a plurality of different terminal (server) groups, for example, a data center. The computing cluster can include a plurality of computing nodes, and the computing power of different computing nodes can be the same or different. The computing power of a computing node is the sum of all callable computing resources of the computing node, which can include a central processing unit (CPU), memory, etc. In actual application scenarios, one or more computing nodes (which can be a physical computer or a virtual computing system) are allocated for a computing task (such as a database query task). In theory, one or more computing nodes will run one or more task shards (subtasks) of the computing task, but in practice, not all task shards may be allocated to "effective" computing nodes. For example, although one or more task shards of the computing task are allocated to one or more computing nodes in a logical sense, in practice, the available computing resources in the computing cluster do not support so many computing nodes, and many computing nodes do not have corresponding computing resources configured and are ineffective; or, although one or more task shards of the computing task are allocated to computing nodes and the computing nodes are allocated corresponding computing resources in the initial task allocation link, due to hardware / software errors, etc., some computing nodes cannot operate normally, so these computing nodes are actually ineffective. Therefore, a scheduling unit needs to be used to allocate a computing task to a computing node with sufficient computing power for execution according to the computing resource usage (such as central processing unit occupancy, memory occupancy, etc.) of each computing node in the current computing cluster. In the environment of the database access system applied in the present disclosure, the database management server can be a computing node of the computing cluster.

[0063] Computing device: a device with computing or processing capability, which can be embodied in the form of a terminal, such as an Internet of Things device, a mobile terminal, a desktop computer, a laptop computer, etc., or a server or a cluster of servers. In the context of the database access system to which the present disclosure applies, the computing device can be a database management server.

[0064] Application scenarios of the present disclosure

[0065] The present disclosure proposes a database management scheme. The entire database management scheme is relatively general and can be used for various hardware devices of continuous query databases, for example, for data centers, for AI (artificial intelligence) acceleration units, for GPUs (graphics processing units), for IOT (Internet of Things) devices capable of executing deep learning models, embedded devices, etc. The database management scheme is independent of the hardware in which the processing unit executing the database management scheme is finally deployed.

[0066] Database access system

[0067] Figure 1 An internal structure diagram of a database access system to which the embodiments of the present disclosure are applied is shown. In some embodiments, as shown in Figure 1 the database access system 100 includes a client 101, a database management server 102, and a database 103.

[0068] In some embodiments, the database 103 is a repository that is built on computer storage devices, and organizes, stores and manages data in certain data structures. In the embodiments of the present disclosure, the implementation form of the computer storage devices that carry the database 103 is not limited, nor is the type of the database 103 and the data structure it adopts. For example, from the data structure, the database 103 can be a hierarchical database, a network database or a relational database; from the storage format, the database 103 can be a row database or a column database; from the database language, the database 103 can be an SQL database or an Oracle database, etc. In some embodiments, the database management server 102 is a core service for storing, processing and protecting data, and its tasks can include processing transactions related to the database 103, such as designing and creating the database 103, saving various lists and documents required by the database 103, etc., providing daily management support for the database 103 to optimize the performance of the database 103, and responding to the access request of the client 101 to perform relevant operations on the database 103 and return the operation results to the client 101, thereby realizing the viewing, deletion, modification, addition, etc. of data in the database 103. The database management server 102 can be a computing node in a computing cluster (e.g., a data center), and the computing power of the database management server 102 is the sum of all callable computing resources of the database management server 102. In some embodiments, a scheduling unit can be used to distribute database query tasks to the database management server 102 with sufficient computing power for execution according to the computing resource usage (e.g., central processing unit occupancy, memory occupancy, etc.) of each computing node in the current computing cluster. In some embodiments, the client 101 and the database management server 102 are connected in communication, and the communication connection can be a wired or wireless network connection. In some embodiments, the client 101 and the database management server 102 can be in the same local area network, or in different local area networks. In addition, in some embodiments, the database management server 102 and the database 103 are also connected in communication, and the communication connection can be a wired or wireless network connection. In some embodiments, in terms of deployment implementation, the database management server 102 and the database 103 can be deployed on the same physical device for implementation, or on different physical devices for implementation. When the database management server 102 and the database 103 are deployed on different physical devices for implementation, they can be deployed in the same local area network, or in different local area networks.

[0069] In some embodiments, the client 101 can be regarded as an interactive interface provided by the database 103 to the user, allowing the user to access the database 103 through the client 101. In some embodiments, the client 101 can include any type of device or application configured to interact with the database management server 102. For example, the application refers to a computer program that can provide various specific functions, including but not limited to a billing application, an Internet browser, a multimedia player, etc. When the user needs to access the database 103, or the client 101 has a database access requirement, the client 101 can send a database access request to the database management server 102; the database management server 102 can perform corresponding operations on the database 103 according to the database access request. In some embodiments, in a database query scenario, the client 101 can send a query statement to the database management server 102. According to the different database languages supported by the database 103, the query statement can be different, for example, it can be a SQL statement, or it can be an Oracle statement. In some embodiments, the client 101 can continuously send multiple query statements to the database management server 102, and the multiple query statements are usually the same or similar. The database management server 102 can generate multiple first candidate query plans according to the current query statement, select an intermediate target query plan from the multiple first candidate query plans according to the execution cost of the multiple first candidate query plans, and in the case that the predicted execution time of each query step in the multiple query steps of the intermediate target query plan and the computing resources are sufficient, the second candidate query plan can be generated from the multiple first candidate query plans according to the first data reaching the database 103 in the relevant data queried by the current query statement, and the target query plan is selected as the final execution plan from the multiple first candidate query plans and the second candidate query plan according to the execution cost of the multiple first candidate query plans and the second candidate query plan. The final query plan is executed to query the database 103, to access the database 103 to obtain the relevant data and manipulate the relevant data to generate the query result corresponding to the query, and the query result is returned to the client 101. In the process of continuously querying the database, the target query plan at the current time is found at the multiple predicted execution times in the process of executing the current query statement, which reduces the execution cost of the entire query process of executing the current query statement, thereby improving the database query efficiency. In the case that the current computing resources of the database management server 102 are sufficient, the target query plan at the current time found at the predicted execution time is executed, and in the case that the current computing resources are insufficient, the target query plan at the current time is skipped to execute, which ensures the normal execution of the current query statement, thereby improving the computing resource utilization. Since the process of continuously querying the database 103 by the database management server 102 will be described in detail below, it will not be described here.

[0070] Computing device

[0071] Figure 2 An internal structure diagram of the database management server 102 (computing device 141 or system on a chip 142) of one embodiment of the present disclosure is shown. As shown, the computing device 141 can include one or more processors 22, and a memory 29. The memory 29 in the computing device 141 can be a main memory (referred to as main memory or memory for short) for storing instruction information and / or data information represented by data signals, and can also be used to implement data exchange between the processor 22 and an external storage device 26 (also referred to as auxiliary memory or external memory). Figure 2

[0072] In some cases, the processor 22 can need to access the memory 29 to obtain data in the memory 29 or modify data in the memory 29. Since the access speed of the memory 29 is slow, in order to alleviate the speed gap between the processor 22 and the memory 29, the computing device 141 further includes a cache memory 28 coupled with the bus 21, which is used to cache some program data or message data in the memory 29 that can be repeatedly called. The cache memory 28 is implemented by a storage device such as static random access memory (SRAM) and the like.

[0073] Based on this, the processor 22 can include an instruction execution unit 221, a memory management unit 222, and the like. The instruction execution unit 221 initiates a write access request when executing some instructions that need to modify the memory, the write access request specifying write data to be written into the memory and a corresponding physical address; the memory management unit 222 is used to translate the virtual address specified by these instructions into the physical address mapped by the virtual address, and the physical address specified by the write access request can be consistent with the physical address specified by the corresponding instruction.

[0074] The information interaction between the memory 29 and the cache memory 28 is usually organized in blocks. In some embodiments, the cache memory 28 and the memory 29 can be divided into data blocks according to the same spatial size, and the data blocks can be used as the smallest unit (including one or more data of a preset length) of data exchange between the cache memory 28 and the memory 29. For the sake of brevity and clarity of description, each data block in the cache memory 28 will be referred to as a cache block (which can be referred to as a cacheline or cache line) below, and different cache blocks have different cache block addresses; each data block in the memory 29 will be referred to as a memory block, and different memory blocks have different memory block addresses. The cache block address includes, for example, a physical address tag for locating the data block.

[0075] ​Due to space and resource constraints, the cache memory 28 cannot cache all of the contents of the memory 29, i.e., the storage capacity of the cache memory 28 is typically less than that of the memory 29, and the cache memory 28 provides a limited number of cache block addresses that can correspond to all of the memory block addresses provided by the memory 29. When the processor 22 needs to access memory, it first accesses the cache memory 28 via the bus 21 to determine whether the contents to be accessed are stored in the cache memory 28. If so, the cache memory 28 is hit, and the processor 22 directly calls the contents to be accessed from the cache memory 28. If the contents to be accessed by the processor 22 are not in the cache memory 28, the processor 22 needs to access the memory 29 via the bus 21 to find the corresponding information in the memory 29. Because the access rate of the cache memory 28 is very fast, the efficiency of the processor 22 can be significantly improved when the cache memory 28 is hit, and thus the performance and efficiency of the entire computing device 141 are improved.

[0076] As shown, the processor 22, the cache memory 28, and the memory 29 are packaged in a system on chip (SoC) 201. The SoC architecture can be configured by designers so that the communication between the various elements in the computing device 141 is secure.

[0077] In this example, the computing device 141 can also include various software, including an embedded operating system 203, a loader 202, and applications 204. These software can be fixed in the memory 29 or stored in the external memory 26. Typically, the loader 202 and the embedded operating system 203 are fixed in the memory 29, and the applications 204 can be stored in the external memory 26. In some cases, the loader 202 and the embedded operating system 203 can be combined. For these software, the loader 202 can be configured to verify and load the various software into the cache memory 28. The loader 202 itself can be software that is loaded in a secure manner. The SoC 201 can be configured to retrieve the loader 202 from the memory 29 immediately or soon after the system is powered on or reset, and then determine which software to load based on configuration information, and load the corresponding software into the cache memory 28 based on the verification results for these software, e.g., whether to load a software based on the source, fingerprint, certificate, etc.

[0078] Some of the applications 204 can be independent of the embedded operating system 203 and loaded by the loader 202, and some of the applications 204 can depend on the embedded operating system 203 and loaded and controlled by the embedded operating system 203. As an example, Figure 2Applications 1 through n, n being a natural number other than 0, are shown. Applications 204 can include, without limitation, programs for controlling or responding to external devices (e.g., biometric sensors, printers, microphones, speakers, flow valves, or other I / O components, sensors, actuators, or devices), programs for various I / O tasks, security programs, authentication programs, various computational modules, communication programs, communication support protocols, or other programs, or combinations thereof.

[0079] In some embodiments, the computing device 141 can further include a database management system 30. In some embodiments, the database management system 30 is a functional module in the database management server 102, which can be implemented as a program module of software or as hardware, such as based on FPGA or CPLD, etc. In some embodiments, the database management system 30 can receive a plurality of query statements sent by a client continuously, and can find a target query plan at each of a plurality of predicted execution moments during execution of a current query statement, thereby reducing the execution cost of the entire query process of executing the current query statement, and improving the database query efficiency. In the case where the current computing resources of the database management server 102 are sufficient, the target query plan at the current moment found at each of the predicted execution moments is executed, and in the case where the current computing resources are insufficient, the execution of the target query plan at the current moment is skipped, thereby ensuring the normal execution of the current query statement, which improves the utilization of computing resources. Since the process of continuously querying the database by using the database management system 30 will be described in detail below, further description is omitted here.

[0080] In addition, the computing device 141 can further include a storage device 26, a display device 23, an audio device 24, a mouse / keyboard 25, etc. The storage device 26 is, for example, a hard disk, an optical disk, a flash memory, etc. for information access, which is coupled to the bus 21 through a corresponding interface. The display device 23 is, for example, coupled to the bus 21 through a corresponding display card, and is used for display according to the display signal provided by the bus 21.

[0081] The computing device 141 usually further includes a communication device 27, and thus can communicate with a network or other devices in various ways. The communication device 27 can include one or more communication modules, for example, and the communication device 27 can include a wireless communication module suitable for a specific wireless communication protocol.

[0082] Of course, different computer systems can have different structures according to different mainboards, operating systems and instruction set architectures. For example, many current computer systems are provided with an input / output control center connected between the bus 21 and various input / output devices, and the input / output control center can be integrated into the processor 22 or independent of the processor 22.

[0083] Database management system

[0084] Figure 3 An internal structure diagram of a database management system 30 according to an embodiment of this disclosure is shown. Figure 3 As shown, the database management system 30 includes a job management unit 310, a scheduling unit 320, a metadata management unit 330, a rule unit 340, a search space exploration unit 350, a query optimization unit 360, an execution unit 370, and a storage unit 380. In some embodiments, the database management system 30 can continuously query the database using the same or similar query statements.

[0085] In some embodiments, such as Figure 3 As shown, the job management unit 310 can receive a registration request from the client for the database query job corresponding to the current query statement, and register the database query job in the job management unit 310. The registration information may include the current query statement, the data dependencies between the database query job and other database query jobs on the database query production line, and the termination time of the database query job. In some embodiments, the scheduling unit 320 can formulate a scheduling strategy based on the registration information of the database query job corresponding to the current query statement, or it can formulate a scheduling strategy in a user-defined manner. For example, the scheduling unit 320 can trigger the execution unit 370 to execute the target query plan corresponding to the current query statement when the relevant data queried by the current query statement stored in the database exceeds a certain threshold.

[0086] In some embodiments, the scheduling unit 320 may send a first scheduling request after the database query job corresponding to the current query statement has been successfully registered. In some embodiments, such as Figure 3As shown, the search space exploration unit 350 can parse the semantics and syntax of the current query statement (e.g., a SQL statement) according to the first scheduling request to generate a plurality of first candidate query plans. The first candidate query plans include a plurality of query steps, and the plurality of first candidate query plans are equivalent query plans, i.e., produce the same output for the same input, but with different operators or orders of operators. In other words, the plurality of first candidate query plans have the same function but different forms of implementation. In some embodiments, the search space exploration unit 350 can perform semantic and syntax parsing on the current query statement to obtain a syntax tree, and then generate a logical execution plan of the current query statement based on the parsed syntax tree, and transform the logical execution plan to obtain the plurality of first candidate query plans. In some embodiments, the query optimization unit 360 can select, from the plurality of first candidate query plans, a query plan whose execution cost satisfies a certain numerical condition as an intermediate target query plan based on the execution cost. For example, a query plan with the minimum, maximum, or a certain numerical value of execution cost can be selected from the plurality of first candidate query plans as the intermediate target query plan.

[0087] In some embodiments, the scheduling unit 320 can preset a plurality of predicted execution times, which are the predicted execution times of each query step in a candidate query plan (e.g., the intermediate target query plan) of the plurality of first candidate query plans. In some embodiments, the scheduling unit 320 can send the second scheduling request at each predicted execution time and when the computing resources are sufficient, and pause sending the second scheduling request when the computing resources are insufficient. In some embodiments, the metadata management unit 330 can obtain, at the predicted execution time and according to the second scheduling request, the usage of the computing resources and the first data of the related data queried by the current query statement arriving at the database. In some embodiments, the rule unit 340 can expand and generate a second candidate query plan based on the plurality of first candidate query plans according to the first data. The second candidate query plan includes a plurality of query steps, and the second candidate query plan is an equivalent query plan of the plurality of first candidate query plans, i.e., produces the same output for the same input, but with different operators or orders of operators. The set of the plurality of first candidate query plans and the second candidate query plan constitutes a search space of the current query statement, which is a subset of all candidate query plans of the current query statement, and a candidate query plan can be determined from the search space as a final query plan for execution by the execution unit. In some embodiments, as shown, the rule unit 340 can generate the second candidate query plan according to the first data and the plurality of first candidate query plans. Figure 4As shown, the query path of the first candidate query plan 1 corresponding to the current query statement consists of input, filter 1, operator 1, filter 3, and output. The query path of the first candidate query plan 2 corresponding to the current query statement consists of input, filter 2, operator 2, and output. The query path of the second candidate query plan corresponding to the current query statement consists of input, filter 1, operator 2, operator 3, filter 3, and output. The first candidate query plan 1, the first candidate query plan 2, and the second candidate query plan will produce the same output for the same input. It should be noted that different operators usually have different running efficiencies, and the first candidate query plan 1, the first candidate query plan 2, and the second candidate query plan include different operators and have different execution orders of the operators, so the execution costs of the first candidate query plan 1, the first candidate query plan 2, and the second candidate query plan are different. In some embodiments, as shown in FIG. 3, the database management server can send the first candidate query plan 1, the first candidate query plan 2, and the second candidate query plan to the client, and the client can select the second candidate query plan from the first candidate query plan 1, the first candidate query plan 2, and the second candidate query plan according to the execution cost of each candidate query plan. Figure 3 As shown, the rule unit 340 can include a query step selection module 341 and a query plan expansion module 342. The query step selection module 341 can select a third query step taking the first data as input data from the first candidate query plan. The query plan expansion module 342 can adjust the operators and the execution order of the operators for executing the third query step to generate a second candidate query plan. In some embodiments, the client sends the same query statement to the database management server every certain query period to query the statistical value of a specific event in a past preset time period. For example, the client sends the same query statement to the database management server every 1 day to query the total sales in the past 5 days. For the tthday, the sales of each day from the tthday to the t-4thday need to be queried, and then the sales of each day are summed to obtain the query result. The first candidate query plan corresponding to the query statement can be:

[0088] SUM(t-4)+SUM(t-3)+SUM(t-2)+SUM(t-1)+SUM(t) (1)

[0089] wherein SUM(t) is the sales of the tthday, and the sales data corresponding to SUM(t) arrives at the database on the tthday. For example, on the t-2thday, the first data stored in the database includes the sales data corresponding to SUM(t-4), the sales data corresponding to SUM(t-3), and the sales data corresponding to SUM(t-2). The third query step taking the first data as input data is SUM(t-4), SUM(t-3), and SUM(t-2). The third query step SUM(t-4), SUM(t-3), and SUM(t-2) can be adjusted to SUM((t-4)→(t-2)), and the generated second candidate query plan can be

[0090] SUM((t-4)→(t-2))+SUM(t-1)+SUM(t) (2)

[0091] where SUM((t-4)→(t-2)) is the sales from day t-4 to day t-2.

[0092] In some embodiments, the query optimization unit 360 can estimate the execution cost of the plurality of first candidate query plans and the plurality of second candidate query plans at the predicted execution time and when the computing resources are sufficient, and select a target query plan at the predicted execution time based on the execution cost. In some embodiments, the query optimization unit 360 can compare the current computing resource occupation (e.g., processor occupation and memory occupation) of the database management server with a preset threshold, and determine that the current computing resources are sufficient if the current computing resource occupation is less than the preset threshold, or determine that the current computing resources are insufficient otherwise. In some embodiments, the plurality of first candidate query plans and the plurality of second candidate query plans can include different query steps. For example, a certain current query statement involves three tables, and the connection order of the three tables and / or the operators used to implement the current query plan are different in different candidate query plans, and the actual execution cost is also different, so the target query plan can be determined by evaluating the execution cost of each candidate query plan. In some embodiments, the query optimization unit 360 can select a query plan whose execution cost satisfies a certain numerical condition as the target query plan at the predicted execution time based on the execution cost of the plurality of first candidate query plans and the plurality of second candidate query plans when the computing resources are sufficient at the predicted execution time. For example, the query plan with the minimum, maximum, or certain execution cost can be selected from the plurality of first candidate query plans and the plurality of second candidate query plans as the target query plan. Since the execution cost estimation method of the query plan is prior art, it is not described in detail. In some embodiments, for the target query plan at each predicted execution time, the execution unit 370 executes the target query plan corresponding to the current query statement based on the query results of the query steps of the target query plan corresponding to the last query statement in an incremental manner to generate the query result corresponding to the current query statement. In some embodiments, the execution unit 370 receives the target query plan corresponding to the current query statement, and executes the query plan according to the query steps in the target query plan. Each target query plan has a plurality of corresponding query steps, and each query step is executed by an operator in the execution unit 370 (one operator is responsible for completing one basic data processing logic, and a group of operators complete a group of query steps of data according to the target query plan, in other words, one query step is executed by at least one operator and the corresponding query result is obtained). In some embodiments, the main operators in the execution unit 370 include data reading operation, conditional filtering operation, connection operation, set operation, grouping (Group by) operation, de-duplication (Distinct) operation, aggregation operation, and sorting operation, etc. The conditional filtering operation can be for a data table or for a subquery, and can also have a filtering condition; the set operation includes intersection, union, difference, etc.The execution unit 370 can encapsulate the plurality of query steps belonging to the target query plan as the query result of the target query plan, i.e., the query result of the target query plan includes the query results of the plurality of query steps. Therefore, in actual application, the query result of the target query plan corresponding to the current query statement can be obtained from the execution unit 370 in units of the target query plan. In some embodiments, as shown in FIG. 3B, the execution unit 370 can encapsulate the query result of the target query plan as the query result of the current query statement. Figure 5 As shown in FIG. 3B, the relevant data queried by the current query statement gradually arrives at the database over time, wherein the relevant data arriving at the database is the first data, and the usage of the computing resources of the database management server also changes over time. t1, t2, t3, t4 and t5 are the predicted execution time, wherein the current computing resource is insufficient at the predicted execution time t1, and the current computing resource is sufficient at the predicted execution time t2, t3, t4 and t5. Therefore, at the predicted execution time t1, the query optimization unit 360 skips generating the target query plan of the current time, and the execution unit 370 skips executing the target query plan of the current time. At the predicted execution time t2, t3, t4 and t5, the query optimization unit 360 generates the target query plan of the current time respectively, and the execution unit 370 executes the target query plan of the current time respectively, thereby obtaining the query result of the current query statement.

[0093] In some embodiments, the client continuously sends a plurality of query statements to the database management server, and the plurality of query statements are generally the same or similar, which makes it possible that the plurality of query steps of the target query plan corresponding to the current query statement can include the query steps executed in the target query plan corresponding to the previous query statement in the process of continuously querying the database, i.e., the target query plan corresponding to the current query statement and the target query plan corresponding to the previous query statement can include the same query steps. It should be noted that the same query steps can have the same input and output and execute the same data processing logic. In some embodiments, the plurality of query steps of the target query plan corresponding to the current query statement include a first query step and a second query step, wherein the first query step is a query step executed in the target query plan corresponding to the previous query statement, and the second query step is a query step not executed in the target query plan corresponding to the previous query statement. The storage unit 380 can store the first query result obtained by executing the first query step in the target query plan corresponding to the previous query statement. In some embodiments, in the case that the database management system 30 regularly and continuously queries the database with the same or similar query statements, the storage unit 380 can determine the first query result to be stored and the storage duration thereof based on the query period and the data read by each query statement when querying the database.

[0094] In some embodiments, as shown in FIG. 3B, the execution unit 370 can encapsulate the query result of the target query plan as the query result of the current query statement. Figure 3As shown, the query optimization unit 360 includes a first reading module 361, an execution cost estimation module 362, and a query plan selection module 363. For each target query plan at each predicted execution time, the first reading module 361 can read the first query result when the first query step needs to be executed in the execution cost estimation process. The execution cost estimation module 362 can estimate the execution cost of the plurality of first candidate query plans and second candidate query plans corresponding to the current query statement based on the first query result using an execution cost estimation method, and read the first query result and skip the first query step when the first query step needs to be executed in the execution cost estimation process. Since the execution cost estimation method is prior art, it is not described in detail. The query plan selection module 363 can select the query plan with the minimum execution cost from the plurality of first candidate query plans and second candidate query plans as the target query plan (i.e., the optimal query plan) corresponding to the current query statement. In some embodiments, as shown in FIG. 3B, the query plan selection module 363 can include a first query plan selection module 361a and a second query plan selection module 361b. The first query plan selection module 361a can select the query plan with the minimum execution cost from the plurality of first candidate query plans as the target query plan corresponding to the current query statement. The second query plan selection module 361b can select the query plan with the minimum execution cost from the plurality of second candidate query plans as the target query plan corresponding to the current query statement. Figure 3 As shown, the execution unit 370 includes a second reading module 371, a query plan execution module 372, and a query result generation module 373. For each target query plan at each predicted execution time, the second reading module 371 can read the first query result when the first query step needs to be executed in the execution process of the target query plan corresponding to the current query statement. The query plan execution module 372 is configured to execute a plurality of query steps according to the target query plan corresponding to the current query statement, wherein the second query result is obtained by executing the second query step, and the first query result is read and the first query step is skipped when the first query step needs to be executed in the execution process of the target query plan corresponding to the current query statement. The query result generation module 373 can generate the query result corresponding to the current query statement based on the first query result and the second query result. For example, the generated first query result and second query result at each predicted execution time can be added or aggregated according to the incremental query plan model based on the time-varying relationship to generate the query result corresponding to the current query statement. In this way, for each target query plan at each predicted execution time, when the first query step that has been executed in the target query plan corresponding to the previous query statement needs to be executed in the target query plan corresponding to the current query statement to obtain the first query result, the first query step does not need to be repeatedly executed, and the query result corresponding to the current query statement can be generated based on the read first query result and the second query result obtained by executing the second query step, thereby improving the database query efficiency and reducing the computational overhead of the database query.

[0095] Database management method according to embodiments of the present disclosure

[0096] According to one embodiment of the present disclosure, a database management method is provided. The method can be performed by a database management system 30. In the case that the computing device 141 is a single computer, the database management system 30 is part of the single computer, and the database management method is performed by the part of the single computer. In the case that the computing device 141 is a collection of multiple computers, the database management system 30 is a single computer, and the database management method is performed by the single computer. In the case that the computing device 141 is in the form of a cloud, the database management system 30 is a series of computers or parts of computers in the cloud, and the database management method is performed by the series of computers or parts of computers in the cloud.

[0097] As shown in FIG. 6, the database management method according to one embodiment of the present disclosure includes the following steps. Figure 6 S610, parsing semantics and syntax of a current query statement according to a first scheduling request to generate a plurality of first candidate query plans; S620, obtaining usage of computing resources and first data arriving at a database from related data queried by the current query statement according to a second scheduling request at a predicted execution time, the predicted execution time being a predicted execution time of each query step in a query plan of the plurality of first candidate query plans; S630, expanding a second candidate query plan based on the plurality of first candidate query plans according to the first data; and S640, selecting a target query plan from the plurality of first candidate query plans and the second candidate query plan based on execution cost at the predicted execution time and in the case that computing resources are sufficient, wherein the database is queried by executing the target query plan.

[0098] Since the implementation details of the above database management method have been described in the detailed description of the device embodiments above, they will not be described here to save space.

[0099] Business value of the present disclosure

[0100] In the scenario of the continuous query database, the database management server can receive a plurality of query statements continuously sent by the client, and can find the target query plan at the current time at a plurality of predicted execution moments in the execution of the current query statement, so as to reduce the execution cost of the entire query process of the current query statement, thereby improving the database query efficiency. The target query plan at the current time found at the predicted execution moment is executed in the case that the current computing resource of the database management server is sufficient, and the target query plan at the current time is skipped in the case that the current computing resource is insufficient, so as to ensure the normal execution of the current query statement, which improves the computing resource utilization. In this scenario, the computing resource utilization of the computing device is improved, thereby reducing the operation cost of the computing device. The disclosed embodiment reduces the operation cost of the computing device, and thus has good commercial value and economic value.

[0101] Those skilled in the art can understand that the present disclosure can be implemented as a system, a method and a computer program product. Therefore, the present disclosure can be embodied in the form of entire hardware, entire software (including firmware, resident software, microcode), and a combination of software and hardware. In addition, in some embodiments, the present disclosure can also be embodied in the form of a computer program product in one or more computer readable media, which contains computer readable program codes.

[0102] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium, for example, but not limited to, is an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium include: electrical connection of a specific one or more wires, portable computer disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage, magnetic storage or any suitable combination of the above. In this paper, the computer readable storage medium can be any tangible medium containing or storing programs, which can be used by or in combination with processing units, devices or apparatuses.

[0103] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that can be involved in

[0104] The computer readable medium can include the computer readable storage medium or the computer readable signal medium. The computer readable storage medium can include any tangible storage media. The computer readable signal medium can include any computer readable medium for transmitting or receiving computer readable instructions.

[0105] The computer program code can be implemented in any of a variety of programming languages. Such programming languages include, among others, Java, C++, and C. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0106] The specific embodiments of the present disclosure have been shown and described in detail, to simply explain and understand the present disclosure. The present disclosure can be variously modified and changed by those skilled in the art without departing from the spirit and scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims

1. A database management system, comprising: a search space exploration unit configured to receive a first scheduling request after a database query job corresponding to a current query statement is successfully registered, and to parse semantics and syntax of the current query statement according to the first scheduling request, and to generate a plurality of first candidate query plans; a metadata management unit configured to receive a second scheduling request at a predicted execution time and when computing resources are sufficient, and to obtain a usage of the computing resources and first data arriving at a database from related data queried by the current query statement according to the second scheduling request, the predicted execution time being a predicted execution time of each query step in a query plan of the plurality of first candidate query plans; a rule unit configured to generate a second candidate query plan based on the plurality of first candidate query plans according to the first data; a query optimization unit configured to select a target query plan from the plurality of first candidate query plans and the second candidate query plan based on execution cost at the predicted execution time and when the computing resources are sufficient, wherein the database is queried by executing the target query plan.

2. The database management system of claim 1, wherein, Further comprising: an execution unit configured to execute the target query plan corresponding to the current query statement in an incremental query step manner based on a query result of a query step of a target query plan corresponding to a previous query statement, to generate a query result corresponding to the current query statement.

3. The database management system of claim 2, wherein, The target query plan corresponding to the current query statement comprises a plurality of query steps, the plurality of query steps comprising a first query step and a second query step, the first query step being a query step that has been executed in a target query plan corresponding to a previous query statement, and the second query step being a query step that has not been executed in the target query plan corresponding to the previous query statement, the database management system further comprising: a storage unit configured to store a first query result obtained by executing the first query step in the target query plan corresponding to the previous query statement.

4. The database management system of claim 3, wherein, The query optimization unit comprises a first reading module, an execution cost estimation module, and a query plan selection module, the first reading module is configured to read the first query result when the first query step needs to be executed; the execution cost estimation module is configured to estimate execution cost of the plurality of first candidate query plans and the second candidate query plan using an execution cost estimation method based on the first query result, wherein the first query result is read and the first query step is skipped during the execution cost estimation; the query plan selection module is configured to select a query plan with minimum execution cost from the plurality of first candidate query plans and the second candidate query plan as the target query plan.

5. The database management system of claim 1, wherein, The first candidate query plan comprises a plurality of query steps, and the rule unit comprises a query step selection module and a query plan expansion module, the query step selection module is configured to select a third query step taking the first data as input data from the first candidate query plan; the query plan expansion module is configured to expand the first candidate query plan based on the third query step to generate the second candidate query plan. The query plan expansion module is configured to adjust an operator and an operator execution order of the third query step to generate the second candidate query plan.

6. The database management system of claim 3, wherein, The execution unit comprises a second reading module, a query plan execution module and a query result generation module, The second reading module is configured to read the first query result when the first query step needs to be executed. The query plan execution module is configured to execute the plurality of query steps according to the target query plan corresponding to the current query statement, wherein the second query result is obtained by executing the second query step, and the first query step is skipped by reading the first query result. The query result generation module is configured to generate a query result corresponding to the current query statement based on the first query result and the second query result.

7. The database management system of claim 1, wherein, Further comprising: A job management unit configured to register a database query job corresponding to the current query statement; A scheduling unit configured to send the first scheduling request after the database query job corresponding to the current query statement is successfully registered.

8. The database management system of claim 7, wherein, The predicted execution time comprises a plurality of predicted execution times, and the scheduling unit is further configured to preset the plurality of predicted execution times and send the second scheduling request at the predicted execution time and when computing resources are sufficient.

9. A database management engine comprising: A search space exploration unit configured to receive a first scheduling request after a database query job corresponding to a current query statement is successfully registered, and parse semantics and syntax of the current query statement according to the first scheduling request to generate a plurality of first candidate query plans; A metadata management unit configured to receive a second scheduling request at a predicted execution time and when computing resources are sufficient, and according to the second scheduling request, obtain a usage of the computing resources and first data reaching a database from related data queried by the current query statement, the predicted execution time being a predicted execution time of each query step in a plurality of query steps of one query plan in the plurality of first candidate query plans; A rule unit configured to generate a second candidate query plan based on the plurality of first candidate query plans according to the first data; A query optimization unit configured to select a target query plan from the plurality of first candidate query plans and the second candidate query plan based on execution cost when the predicted execution time and the computing resources are sufficient, wherein the database is queried by executing the target query plan.

10. A computing device comprising: a memory configured to store computer executable code; a processor configured to execute the computer executable code; and the database management system of any one of claims 1-8.

11. A system on chip comprising: a memory configured to store computer executable code; a processor configured to execute the computer executable code; and the database management system of any one of claims 1-8.

12. A database management method comprising: ​ ​ receive a first scheduling request after a database query job corresponding to the current query statement is successfully registered, and parse semantics and syntax of the current query statement according to the first scheduling request to generate a plurality of first candidate query plans; at a predicted execution time and in a case where computing resources are sufficient, receive a second scheduling request, and according to the second scheduling request, obtain a usage of the computing resources and first data of related data queried by the current query statement and arriving at a database; based on the first data, expand the plurality of first candidate query plans to generate second candidate query plans; at the predicted execution time and in the case where the computing resources are sufficient, select a target query plan from the plurality of first candidate query plans and the second candidate query plans based on an execution cost, and query the database by executing the target query plan.

13. A computer readable medium comprising computer executable code that, when executed by a processor, implements the method of claim 12.

Images