Data query method and device for distributed database, equipment, storage medium and program product

Abstract

The invention discloses a data query method and device for a distributed database, equipment, a storage medium and a program product, and the method comprises the steps: responding to a query request, analyzing a query statement in the query request, and generating a query plan; the query plan comprises a plurality of query operators, and each query operator corresponds to a query stage of the data query operation; based on the characteristics of each query operator and the data distribution information of the distributed database, determining candidate operators from a plurality of query operators contained in the query plan; determining a target operator from the candidate operators based on a preset push-down condition; the target operator is an operator capable of executing query in a storage engine of a distributed database; based on the target operator, optimizing the query plan to generate an optimized query plan; a target operator in the optimized query plan is pushed down to the storage engine to execute query, and other operators execute query in the query engine. By adopting the method, the performance of the distributed database can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data query method, device, equipment, storage medium and program product for a distributed database. Background Art

[0002] With the rapid growth of business data, traditional databases cannot effectively meet the requirements of data computing and processing, and distributed databases are urgently needed to solve problems such as compatibility, reliability and performance. However, the performance of distributed databases is low.

[0003] Therefore, how to improve the performance of distributed databases has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a data query method, apparatus, device, storage medium and program product for a distributed database, which can improve the performance of the distributed database.

[0005] In a first aspect, an embodiment of the present application provides a data query method for a distributed database, which is applied to a database management node in a distributed cluster where a distributed database is deployed; the method includes:

[0006] In response to a query request, the query statement in the query request is parsed to generate a query plan; the query plan includes multiple query operators, each query operator corresponds to a query stage of the data query operation;

[0007] Based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from multiple query operators included in the query plan;

[0008] Based on the preset push-down conditions, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database;

[0009] Based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators are executed in the query engine.

[0010] In one of the embodiments, the data distribution information includes the actual storage location of the data, the computing resources of each working node, and the network resources; based on the characteristics of each query operator and the data distribution information of the distributed database, candidate operators are determined from multiple query operators included in the query plan, including: based on the characteristics of each query operator, multiple operators with target index, continuous order or aggregation capabilities are determined from multiple query operators included in the query plan; based on the actual storage location of the data, the computing resources of each working node, and the network resources, candidate operators are determined from multiple operators.

[0011] In one of the embodiments, based on preset push-down conditions, a target operator is determined from candidate operators, including: determining at least one operator from the candidate operators that satisfies the constraints of the storage engine; the constraints include query syntax or query operations supported by the storage engine; among the at least one operator determined, if any operator is pushed down to the storage engine to execute the query and the query result obtained is correct and complete, any operator is used as the target operator.

[0012] In one of the embodiments, a query plan is optimized based on a target operator to generate an optimized query plan, including: selecting a target transaction management protocol from a plurality of distributed transaction management protocols based on the characteristics of the query request and business requirements; optimizing the target transaction management protocol to obtain an optimized distributed transaction protocol; optimizing the query plan based on the optimized distributed transaction protocol and the target operator to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine through the optimized distributed transaction protocol.

[0013] In one of the embodiments, the method further includes: performing data slicing processing on metadata in a distributed database to obtain a plurality of data segments; and determining data distribution information of the distributed database based on the plurality of data segments.

[0014] In one of the embodiments, the method further includes: real-time monitoring of the load of each working node in the distributed cluster and the data distribution corresponding to the data in the distributed database deployed on the distributed cluster; when it is determined that the load of any working node is greater than a preset load threshold, or the data is unevenly distributed, returning to execute the step of performing data slicing processing on the metadata in the distributed database to obtain multiple data fragments, until the load of any working node is less than or equal to the preset load threshold, or the data distribution is uniform.

[0015] In a second aspect, the present application provides a data query device for a distributed database, which is applied to a database management node in a distributed cluster where a distributed database is deployed; the device includes:

[0016] A parsing module, which is used to parse the query statement in the query request in response to the query request and generate a query plan; the query plan includes multiple query operators, each query operator corresponds to a query stage of the data query operation;

[0017] A determination module, used to determine a candidate operator from multiple query operators included in the query plan based on the characteristics of each query operator and data distribution information of the distributed database;

[0018] The determination module is further used to determine a target operator from the candidate operators based on a preset push-down condition; the target operator is an operator that can execute a query in the storage engine of the distributed database;

[0019] The optimization module is used to optimize the query plan based on the target operator and generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators are executed in the query engine.

[0020] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0021] In response to a query request, the query statement in the query request is parsed to generate a query plan; the query plan includes multiple query operators, each query operator corresponds to a query stage of the data query operation;

[0022] Based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from multiple query operators included in the query plan;

[0023] Based on the preset push-down conditions, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database;

[0024] Based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators are executed in the query engine.

[0025] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0026] In response to a query request, the query statement in the query request is parsed to generate a query plan; the query plan includes multiple query operators, each query operator corresponds to a query stage of the data query operation;

[0027] Based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from multiple query operators included in the query plan;

[0028] Based on the preset push-down conditions, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database;

[0029] Based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators are executed in the query engine.

[0030] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0031] In response to a query request, the query statement in the query request is parsed to generate a query plan; the query plan includes multiple query operators, each query operator corresponds to a query stage of the data query operation;

[0032] Based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from multiple query operators included in the query plan;

[0033] Based on the preset push-down conditions, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database;

[0034] Based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators are executed in the query engine.

[0035] The above-mentioned distributed database data query method, device, equipment, storage medium and program product, the database management node can respond to the query request, parse the query statement in the query request, and generate a query plan; the query plan contains multiple query operators, each query operator corresponds to a query stage of the data query operation; based on the characteristics of each query operator and the data distribution information of the distributed database, candidate operators are determined from the multiple query operators contained in the query plan; based on the preset push-down conditions, the target operator is determined from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database; based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine. By adopting this method, the database management node can determine candidate operators from multiple query operators included in the query plan based on the characteristics of each query operator included in the query plan and the data distribution information of the distributed database, and based on preset push-down conditions, determine the target operator that can be pushed down to the storage engine to execute the query from the candidate operators. After that, the query plan is optimized based on the target operator to generate an optimized query plan. In this way, by executing the target operator in the optimized query plan in the storage engine and other operators executing the query in the query engine, the high performance and high concurrency capabilities of the distributed database can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 This is a schematic diagram of an application scenario of a distributed database data query method provided in an embodiment of the present application;

[0038] Figure 2 It is a flow chart of a data query method of a distributed database provided in an embodiment of the present application;

[0039] Figure 3 It is a flow chart of another distributed database data query method provided in an embodiment of the present application;

[0040] Figure 4 It is a structural diagram of a data query device for a distributed database provided in an embodiment of the present application;

[0041] Figure 5 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] The following introduces the application scenarios of the data query method of the distributed database provided in the embodiment of the present application.

[0044] See also Figure 1 , Figure 1 Schematic diagram of an application scenario of a distributed database data query method provided in an embodiment of the present application. Figure 1 As shown, the distributed cluster 100 includes a database management node 101 , a storage engine 102 and a query engine 103 .

[0045] Among them, the database management node 101 can respond to the query request, parse the query statement in the query request, and generate a query plan; the query plan contains multiple query operators, each query operator corresponds to a query stage of the data query operation; based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from the multiple query operators contained in the query plan; based on the preset push-down conditions, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database; based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine 102 to execute the query, and other operators execute the query in the query engine 103. By adopting this method, the database management node 101 can determine candidate operators from multiple query operators included in the query plan based on the characteristics of each query operator included in the query plan and the data distribution information of the distributed database, and based on the preset push-down conditions, determine the target operator from the candidate operators that can be pushed down to the storage engine to execute the query. Thereafter, the query plan is optimized based on the target operator to generate an optimized query plan. In this way, by executing the target operator in the optimized query plan in the storage engine and other operators executing queries in the query engine, the high performance and high concurrency capabilities of the distributed database can be improved.

[0046] Optionally, the database management node 101 may be a server. The server mentioned here may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers.

[0047] See also Figure 2 , Figure 2 1 is a flow chart of a data query method for a distributed database provided in an embodiment of the present application. The method can be executed by a database management node (for example, the above-mentioned database management node 101) in a distributed cluster. Figure 2 As shown, the data query method of the distributed database may include but is not limited to the following steps:

[0048] S201. In response to a query request, a query statement in the query request is parsed to generate a query plan; the query plan includes multiple query operators, each query operator corresponding to a query stage of a data query operation.

[0049] In an optional implementation, before step S201, the database management node may also receive a query request from the application layer or the user end.

[0050] In an optional implementation, the query statement in the query request may be an SQL statement, and a SQL parser may be deployed in the database management node; the database management node parses the query statement in the query request and generates a query plan, which may be that the database management node parses the SQL query statement in the query request through the SQL parser to generate a query plan. Optionally, the database management node may also parse the SQL query statement in the query request through the SQL parser to generate a query tree.

[0051] S202: Based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from multiple query operators included in the query plan.

[0052] Among them, a distributed database is a logically unified database formed by connecting multiple physically dispersed database units using a computer network. Each connected database unit is called a site or node. A distributed database has a unified database management system to manage it, called a distributed database management system. The basic characteristics of a distributed database include: physical distribution, logical integrity, and site autonomy. Other characteristics that can be derived from these three basic characteristics include: data distribution transparency, a mechanism for reaching consensus according to established protocols, appropriate data redundancy, and distribution of transaction management.

[0053] Data distribution information: In a distributed database, data is usually divided and stored on different nodes to optimize query performance and resource utilization. The data distribution information of the distributed database can be determined based on the data corresponding to each node and the node resource utilization. Optionally, the data distribution information can be determined based on strategies such as hash distribution and range distribution, which are not limited here.

[0054] Among them, hash distribution and range distribution are two different data distribution strategies, which are used to optimize data storage and access in database management and distributed systems. Hash distribution is a common data distribution strategy, which distributes data to different nodes or storage units through hash functions. Its main advantage is that data is evenly distributed, which can effectively reduce the load imbalance caused by hot data. However, hash distribution also has its limitations, especially when dealing with changes in data volume, which may cause a large amount of data migration. Range distribution distributes data to different nodes according to a certain order (such as numerical size). Its advantage is that it can ensure the physical continuity of data, which is very beneficial for range queries. However, range distribution may lead to uneven data distribution, especially when the data volume grows unevenly.

[0055] In an optional implementation, the data distribution information of the distributed database may include but is not limited to the storage location of the data in the distributed database on each node, sharding rules, etc. Optionally, the data distribution information of the distributed database may be determined by the data management module and metadata service inside the distributed database system.

[0056] Optionally, the candidate operators include but are not limited to operators that can directly utilize the internal indexing, sorting, or aggregation capabilities of the storage engine.

[0057] S203. Based on a preset push-down condition, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in a storage engine of a distributed database.

[0058] In an optional implementation, the database management node may use the operator that meets the push-down condition among the candidate operators as the target operator. Optionally, the push-down condition may include: (1) the operator meets the constraints or restrictions of the storage engine; (2) the query result of the data query after push-down is correct and complete.

[0059] S204. Based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine.

[0060] In an optional implementation, when the query plan is a SQL execution plan, the optimized query plan may refer to an optimized SQL query plan. The SQL execution plan is generated by a database query optimizer based on a query statement, and describes how to execute the query to obtain the desired result. The optimized SQL execution plan means a SQL execution plan that executes the query in the most efficient way found by the query optimizer, which may include selecting appropriate indexes, connection order, parallel execution strategy, etc.

[0061] In an embodiment of the present application, a database management node may parse a query statement in a query request in response to a query request and generate a query plan; the query plan includes multiple query operators, each query operator corresponding to a query stage of a data query operation; based on the characteristics of each query operator and the data distribution information of a distributed database, a candidate operator is determined from the multiple query operators included in the query plan; based on a preset push-down condition, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in a storage engine of a distributed database; based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine. By adopting this method, the database management node can determine candidate operators from multiple query operators included in the query plan based on the characteristics of each query operator included in the query plan and the data distribution information of the distributed database, and based on preset push-down conditions, determine the target operator that can be pushed down to the storage engine to execute the query from the candidate operators. After that, the query plan is optimized based on the target operator to generate an optimized query plan. In this way, by executing the target operator in the optimized query plan in the storage engine and other operators executing the query in the query engine, the high performance and high concurrency capabilities of the distributed database can be improved.

[0062] See also Figure 3 , Figure 3 1 is a flow chart of another distributed database data query method provided in an embodiment of the present application. Figure 2 The difference between the data query method of the distributed database shown is that Figure 3 The method shown also explains how the database management node optimizes the query plan based on the target operator and generates an optimized query plan. Figure 3 As shown, the data query method of the distributed database may include but is not limited to the following steps:

[0063] S301. In response to a query request, a query statement in the query request is parsed to generate a query plan; the query plan includes multiple query operators, each query operator corresponding to a query stage of a data query operation.

[0064] S302: Based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from multiple query operators included in the query plan.

[0065] In an optional implementation, the data distribution information may include the actual storage location of the data, the computing resources of each working node, and the network resources; the database management node determines the candidate operator from the multiple query operators included in the query plan based on the characteristics of each query operator and the data distribution information of the distributed database, which may include the following steps:

[0066] Step 1: Based on the characteristics of each query operator, multiple operators with target index, continuous order or aggregation capability are determined from multiple query operators included in the query plan;

[0067] Step 2: Based on the actual storage location of the data, the computing resources of each working node, and the network resources, a candidate operator is determined from multiple operators.

[0068] In an optional implementation, before step S302, the database management node may also perform data slicing processing on the metadata in the distributed database to obtain multiple data fragments; based on the multiple data fragments, determine the data distribution information of the distributed database. Optionally, the database management node may also monitor the load of each working node in the distributed cluster and the data distribution corresponding to the data in the distributed database deployed on the distributed cluster in real time; when it is determined that the load of any working node is greater than the preset load threshold, or the data is unevenly distributed, return to perform the step of performing data slicing processing on the metadata in the distributed database to obtain multiple data fragments, until the load of any working node is less than or equal to the preset load threshold, or the data distribution is uniform.

[0069] In an optional implementation, the database management node may also adjust the storage location of data in the distributed database of the database management node based on the load of each working node and the data distribution of the database management node to achieve load balancing.

[0070] S303. Based on the preset push-down condition, determine the target operator from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database.

[0071] In an optional implementation, the database management node determines the target operator from the candidate operators based on preset push-down conditions, which may include: determining at least one operator from the candidate operators that satisfies the constraints of the storage engine; the constraints include the query syntax or query operations supported by the storage engine; among the at least one operator determined, if any operator is pushed down to the storage engine to execute the query and the query result obtained is correct and complete, then any operator is used as the target operator.

[0072] S304: Select a target transaction management protocol from a plurality of distributed transaction management protocols based on the characteristics of the query request and business requirements.

[0073] Among them, the multiple distributed transaction management protocols may include but are not limited to a one-phase commit protocol and a two-phase commit protocol, and the two-phase commit protocol includes an optimistic mode commit protocol and a pessimistic mode commit protocol. The optimistic mode commit protocol means that during the execution of a transaction, other transactions will not modify the data to be accessed by the current transaction. Therefore, the data will not be locked at the beginning of the transaction, but it will be checked whether the data has been modified by other transactions when the transaction is committed. If the data has not been modified, the transaction is successfully committed; if the data has been modified, the transaction is rolled back and may need to be retried. The optimistic transaction mode is suitable for scenarios with more reads and less writes, because it can reduce the use of locks, thereby improving the system concurrency performance. The pessimistic mode commit protocol is opposite to the optimistic transaction commit protocol. The pessimistic transaction commit protocol assumes that during the execution of a transaction, other transactions may modify the data to be accessed by the current transaction. Therefore, the data will be locked at the beginning of the transaction to ensure that other transactions cannot access or modify the data before the transaction is completed. The pessimistic transaction mode is suitable for scenarios with more writes and less reads, because it avoids data conflicts through a locking mechanism.

[0074] In the present application, the database management node selects the target transaction protocol to ensure that the data is either successfully committed or rolled back on all participating nodes to maintain the consistency and integrity of the data, thereby improving the processing performance of the database system.

[0075] S305: Optimize the target transaction management protocol to obtain an optimized distributed transaction protocol.

[0076] In a distributed system, communication between nodes is inevitable, but too much network communication will increase the delay of transaction processing. Therefore, unnecessary communication can be reduced by optimizing the target transaction management protocol, or more efficient communication methods (such as asynchronous communication) can be used to reduce network delays. In addition, in distributed transactions, a single point of failure may cause the entire transaction to fail. Therefore, the target transaction management protocol needs to be optimized. Among them, the database management node optimizes the target transaction management protocol, which is essentially to optimize the transaction commit and rollback mechanism corresponding to the target transaction management protocol.

[0077] In an optional implementation, the database management node may use a variety of fault-tolerant mechanisms, such as redundant nodes, distributed locks, consensus algorithms, etc., to optimize the transaction submission and rollback mechanisms corresponding to the target transaction management protocol to ensure that transactions can still be submitted or rolled back normally in the event of a single point failure. For example, the database management node may back up each data segment to obtain a data segment copy corresponding to each data segment, and back up the metadata to obtain a metadata copy; wherein the metadata is stored in the metadata center or database management node of the distributed cluster; and the metadata copy and multiple data segment copies are managed using distributed locks to block the metadata copy and multiple data segment copies. In this way, it can be determined that when a single point failure occurs, the transaction can still be submitted or rolled back correctly, thereby improving the processing performance of the database system.

[0078] Optionally, the database management node can also perform consistency checks on multiple copies of data across nodes to ensure that data between multiple nodes in a distributed system is consistent.

[0079] S306. Based on the optimized distributed transaction protocol and the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine; in the optimized query plan, the target operator is pushed down to the storage engine through the optimized distributed transaction protocol.

[0080] In an optional implementation, the database management node may also use column storage to optimize the data compression ratio and query performance in the initial storage engine to obtain an optimized storage engine; and use the optimized storage engine as the storage engine for executing queries.

[0081] In an optional implementation, after step S307, the database management node may also determine the data query results; evaluate the data query results, and based on the evaluation results, re-optimize the optimized query plan to obtain a re-optimized query plan, wherein the target operator in the re-optimized query plan is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine.

[0082] In an embodiment of the present application, the database management node can respond to a query request, parse the query statement in the query request, and generate a query plan; the query plan contains multiple query operators, each query operator corresponds to a query stage of the data query operation; based on the characteristics of each query operator and the data distribution information of the distributed database, a candidate operator is determined from the multiple query operators contained in the query plan; based on a preset push-down condition, a target operator is determined from the candidate operators; the target operator is an operator that can execute queries in the storage engine of the distributed database; based on the characteristics of the query request and business requirements, a target transaction management protocol is selected from a variety of distributed transaction management protocols; the target transaction management protocol is optimized to reduce network latency; based on the optimized distributed transaction protocol and the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine; in the optimized query plan, the target operator is pushed down to the storage engine through the optimized distributed transaction protocol. Using this method,

[0083] In an optional implementation, the database management node may also schedule multiple processing engines to work together based on business needs and data real-time requirements to ensure business isolation, data real-time and strong consistency.

[0084] Optionally, the database management node can be based on a multimodal parallel processing framework, implement a business isolation mechanism, and introduce a real-time data synchronization mechanism, so that the system can efficiently handle various complex business scenarios while ensuring the real-time and consistency of the data.

[0085] Optionally, the database management node can also use the Multi-Version Concurrency Control (MVCC) version backtracking function to implement data recovery based on the global state and time point to ensure the consistency and integrity of the restored data.

[0086] In an optional implementation, the database management node may also utilize an intelligent scheduling executor to intelligently analyze and schedule the optimized query plan, thereby achieving non-intrusive automated processing and improving data processing efficiency and system response speed.

[0087] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0088] Based on the same inventive concept, the embodiment of the present application also provides a distributed database data query device for implementing the above-mentioned distributed database data query method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of the data query device for one or more distributed databases provided below can refer to the limitations of the distributed database data query method above, and will not be repeated here.

[0089] See also Figure 4 , Figure 4 Schematic diagram of a distributed database data query device provided in an embodiment of the present application. Figure 4 As shown, the device is applied to a database management node in a distributed cluster where a distributed database is deployed; the device may include but is not limited to:

[0090] The parsing module 401 is used to parse the query statement in the query request in response to the query request and generate a query plan; the query plan includes multiple query operators, each query operator corresponds to a query stage of the data query operation;

[0091] A determination module 402, configured to determine a candidate operator from a plurality of query operators included in the query plan based on the characteristics of each query operator and data distribution information of the distributed database;

[0092] The determination module 402 is further used to determine a target operator from the candidate operators based on a preset push-down condition; the target operator is an operator that can execute a query in the storage engine of the distributed database;

[0093] The optimization module 403 is used to optimize the query plan based on the target operator and generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine.

[0094] In one embodiment, the data distribution information includes the actual storage location of the data, the computing resources of each working node, and the network resources; when the determination module 402 is used to determine the candidate operators from the multiple query operators included in the query plan based on the characteristics of each query operator and the data distribution information of the distributed database, it is specifically used to: determine multiple operators with target index, continuous order or aggregation capabilities from the multiple query operators included in the query plan based on the characteristics of each query operator; determine the candidate operators from the multiple operators based on the actual storage location of the data, the computing resources of each working node, and the network resources.

[0095] In one embodiment, when the determination module 402 is used to determine the target operator from the candidate operators based on the preset push-down conditions, it is specifically used to: determine at least one operator from the candidate operators that satisfies the constraints of the storage engine; the constraints include the query syntax or query operations supported by the storage engine; among the at least one operator determined, if there is any operator that pushes down to the storage engine to execute the query and the query result obtained is correct and complete, then any operator is used as the target operator.

[0096] In one embodiment, when the optimization module 403 is used to optimize the query plan based on the target operator and generate an optimized query plan, it is specifically used to: select a target transaction management protocol from a plurality of distributed transaction management protocols based on the characteristics of the query request and business requirements; optimize the target transaction management protocol to obtain an optimized distributed transaction protocol; optimize the query plan based on the optimized distributed transaction protocol and the target operator to generate an optimized query plan; the target operator in the optimized query plan is pushed down to the storage engine through the optimized distributed transaction protocol.

[0097] In one embodiment, the determination module 402 is further used to perform data slicing processing on the metadata in the distributed database to obtain multiple data segments; based on the multiple data segments, determine the data distribution information of the distributed database.

[0098] In one embodiment, the device also includes a processing module, which is used to: monitor in real time the load of each working node in the distributed cluster and the data distribution corresponding to the data in the distributed database deployed on the distributed cluster; when it is determined that the load of any working node is greater than a preset load threshold, or the data is unevenly distributed, return to execute the step of performing data slicing processing on the metadata in the distributed database to obtain multiple data fragments, until the load of any working node is less than or equal to the preset load threshold, or the data distribution is uniform.

[0099] Each module in the above-mentioned distributed database data query device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the terminal device in the form of hardware, or can be stored in the memory in the terminal device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0100] In an exemplary embodiment, the present application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a data query method for a distributed database is implemented.

[0101] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0102] In an exemplary embodiment, the present application provides a computer device including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned distributed database data query method when executing the computer program.

[0103] In an exemplary embodiment, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above-mentioned distributed database data query method are implemented.

[0104] In an exemplary embodiment, the present application provides a computer program product, including a computer program, which implements the steps in the above-mentioned distributed database data query method when executed by a processor.

[0105] It should be noted that the data involved in this application (including but not limited to query requests, query plans, query operators, data distribution information, candidate operators, target operators, optimized query plans, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0106] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0107] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0108] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data query method for a distributed database, characterized in that: Applied to a database management node in a distributed cluster where a distributed database is deployed; the method comprises: In response to a query request, a query statement in the query request is parsed to generate a query plan; the query plan includes a plurality of query operators, each of which corresponds to a query stage of a data query operation; Determining a candidate operator from a plurality of query operators included in the query plan based on characteristics of each of the query operators and data distribution information of the distributed database; Based on a preset push-down condition, a target operator is determined from the candidate operators; the target operator is an operator that can execute a query in a storage engine of the distributed database; Based on the target operator, the query plan is optimized to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators are executed in the query engine.

2. The method according to claim 1, characterized in that The data distribution information includes the actual storage location of the data, the computing resources of each working node, and the network resources; the determining of the candidate operator from the multiple query operators included in the query plan based on the characteristics of each query operator and the data distribution information of the distributed database includes: Based on the characteristics of each of the query operators, determining multiple operators having a target index, a continuous order, or an aggregation capability from the multiple query operators included in the query plan; Based on the actual storage location of the data, the computing resources of each of the working nodes, and the network resources, a candidate operator is determined from the multiple operators.

3. The method according to claim 1, characterized in that The step of determining a target operator from the candidate operators based on a preset push-down condition includes: Determine at least one operator that satisfies a constraint condition of the storage engine from the candidate operators; the constraint condition includes a query syntax or a query operation supported by the storage engine; When it is determined that among the at least one operator, if any operator is pushed down to the storage engine to execute the query and the query result obtained is correct and complete, the any operator is used as the target operator.

4. The method according to claim 1, characterized in that: The step of optimizing the query plan based on the target operator to generate an optimized query plan includes: Selecting a target transaction management protocol from a plurality of distributed transaction management protocols based on the characteristics of the query request and business requirements; Optimizing the target transaction management protocol to obtain an optimized distributed transaction protocol; Based on the optimized distributed transaction protocol and the target operator, the query plan is optimized to generate an optimized query plan; the target operator in the optimized query plan is pushed down to the storage engine through the optimized distributed transaction protocol.

5. The method according to claim 1, characterized in that The method further comprises: Performing data slicing processing on the metadata in the distributed database to obtain multiple data fragments; Based on the multiple data fragments, data distribution information of the distributed database is determined.

6. The method according to claim 5, characterized in that The method further comprises: Monitor in real time the load of each working node in the distributed cluster and the data distribution corresponding to the data in the distributed database deployed on the distributed cluster; When it is determined that the load of any of the working nodes is greater than the preset load threshold, or the data is unevenly distributed, return to execute the step of performing data slicing processing on the metadata in the distributed database to obtain multiple data fragments, until the load of any of the working nodes is less than or equal to the preset load threshold, or the data is evenly distributed.

7. A data query device for a distributed database, characterized in that: The device is applied to a database management node in a distributed cluster where a distributed database is deployed; the device comprises: A parsing module, configured to parse the query statement in the query request in response to the query request and generate a query plan; the query plan includes a plurality of query operators, each of which corresponds to a query stage of the data query operation; A determination module, configured to determine a candidate operator from a plurality of query operators included in the query plan based on the characteristics of each query operator and the data distribution information of the distributed database; The determination module is further used to determine a target operator from the candidate operators based on a preset push-down condition; the target operator is an operator that can execute a query in the storage engine of the distributed database; An optimization module is used to optimize the query plan based on the target operator to generate an optimized query plan; in the optimized query plan, the target operator is pushed down to the storage engine to execute the query, and other operators execute the query in the query engine.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images