Data query method and related device

Abstract

Disclosed in the present application is a data query method. On the basis of query costs, a master node determines whether to execute a multi-machine parallel query plan, thereby realizing transparency to upper-layer user services and improving query performance. The method of the present application comprises: after a master node acquires a query request, the master node generating a single-machine query plan and a multi-machine parallel query plan on the basis of the query request, and evaluating a query cost of the single-machine query plan and a query cost of the multi-machine parallel query plan; when the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the master node sending, to a target standby node among at least one standby node, a first execution plan executed by the target standby node, wherein the multi-machine parallel query plan comprises the first execution plan executed by the target standby node and a second execution plan executed by the master node; and the master node and the at least one standby node executing the multi-machine parallel query plan to obtain a query result corresponding to the query request.

Description

A data query method and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202510289070.1, filed on March 11, 2025, entitled “A Data Query Method and Related Equipment”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of database technology, specifically to a data query method and related equipment. Background Technology

[0003] As business operations become increasingly complex, the demands on databases also become more diverse. In large-scale query and analysis scenarios, single-machine database systems often face resource constraints. For complex multi-table joins and aggregation calculations, the processing power of a single-machine database system may not be able to meet business needs, resulting in a single point of failure. Therefore, multi-machine parallel computing technology has emerged.

[0004] Multi-machine parallelism typically refers to breaking down a large computing task into multiple smaller subtasks and distributing them across different computing nodes for parallel processing. However, in distributed database systems based on multi-machine parallel computing technology, the choice of distribution keys for each business table presents the challenge of business-level distributed transformation. Furthermore, due to the randomness of business data distribution, changes in data distribution during production use can lead to storage and computational skew as business evolves. In summary, current centralized and distributed database systems cannot achieve transparency to upper-layer business logic for complex queries, resulting in low data query efficiency.

[0005] Therefore, improving data query performance has become an urgent technical problem to be solved. Summary of the Invention

[0006] This application provides a data query method in which the master node decides to execute a multi-machine parallel query plan or a single-machine query plan based on the query cost. No adaptation modifications are required from the user, which is transparent to the upper-layer user business and improves query performance.

[0007] The first aspect of this application provides a data query method applied to a database system, the database system including a master node and at least one backup node, the at least one backup node including a target backup node, the database system being deployed in cloud computing resources managed by a cloud management platform, the method including: the master node obtaining a query request; the master node generating a single-machine query plan and a multi-machine parallel query plan based on the query request; the master node evaluating the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan respectively; if the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the master node sending a first execution plan executed by the target backup node to the target backup node among the at least one backup node, wherein the multi-machine parallel query plan includes a first execution plan executed by the target backup node and a second execution plan executed by the master node; the master node and the at least one backup node executing the multi-machine parallel query plan to obtain the query result corresponding to the query request.

[0008] As can be seen from the above method, for a query request sent by a user's client, the master node generates both a single-machine query plan and a multi-machine parallel query plan based on the query request. It then evaluates the query costs of both the single-machine and multi-machine parallel query plans, comparing their costs. If the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the master node and at least one backup node execute the multi-machine parallel query plan to obtain the query result corresponding to the query request. Therefore, in a centralized database system deployment scenario, the master node decides whether to execute the multi-machine parallel query plan based on the query cost, requiring no adaptation modifications from the user. This achieves transparency to upper-layer user businesses and leverages the advantages of multi-node parallel execution by executing the multi-machine parallel query plan across nodes, thereby improving query performance.

[0009] In one possible implementation of the first aspect, the master node evaluates the query cost of a single-machine query plan and the query cost of a multi-machine parallel query plan, respectively, including: the master node evaluates the query cost of a single-machine query plan and the query cost of a multi-machine parallel query plan based on a cost evaluation model; the master node compares the query cost of a single-machine query plan and the query cost of a multi-machine parallel query plan.

[0010] Therefore, the cost evaluation model can comprehensively evaluate the execution efficiency and execution cost (such as time cost and computing resource cost) of single-machine query plan and multi-machine parallel query plan, thereby calculating the query cost of single-machine query plan and multi-machine parallel query plan respectively, and deciding on the query plan with the best comprehensive execution efficiency and execution cost from the single-machine query plan or multi-machine parallel query plan for execution.

[0011] In one possible implementation of the first aspect, the query cost includes one or more of the following: execution startup cost, data access cost, data movement cost, or data aggregation cost, wherein the execution startup cost is used to indicate the startup cost of the query plan initialization thread and the query task allocation, the data access cost is used to indicate the data computation cost and the data storage cost, the data movement cost is used to indicate the data communication cost, and the data aggregation cost is used to indicate the cost of data execution aggregation to obtain the final result.

[0012] Therefore, the cost assessment model comprehensively evaluates the execution startup cost, data access cost, data movement cost, or data aggregation cost, and can fully assess the various execution costs in the process of executing the query plan, so as to achieve a more comprehensive query cost assessment.

[0013] In one possible implementation of the first aspect, the method further includes: when the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan, the master node executes the single-machine query plan to obtain the query result corresponding to the query request.

[0014] In one possible implementation of the first aspect, the method further includes: the master node obtaining the status information of the target backup node, wherein the status information of the target backup node includes one or more of the following: the computing power information of the target backup node, the network transmission capability information of the target backup node, or the memory overhead information of the target backup node; the master node determining the amount of task to be executed by the target backup node in the first execution plan based on the status information of the target backup node.

[0015] Therefore, by obtaining the status information of the target backup node, the master node can determine the workload of the target backup node in executing the first execution plan, and select a more suitable physical execution method for each backup node executing the multi-machine parallel query node.

[0016] In one possible implementation of the first aspect, the query request includes a first scan operation, which indicates the data range of the data to be scanned. The master node and at least one backup node execute a multi-machine parallel query plan, including: the master node determining the data range of the data to be scanned included in the first scan operation; the master node determining the target data range to be scanned by the target backup node based on the data range of the data to be scanned; the master node sending the target data range to the target backup node; and the target backup node scanning the data corresponding to the target data range.

[0017] Therefore, when the backup node performs a scanning operation, the master node uniformly processes and allocates the data range to be scanned for each backup node from a global perspective, so that the master and backup nodes can perform single-table regional scanning and give full play to the advantages of multi-node parallel execution.

[0018] In one possible implementation of the first aspect, the master node determines the target data range to be scanned by the target backup node based on the data range of the data to be scanned, including: the master node creating and saving the scanning information of the target backup node, wherein the scanning information of the target backup node includes one or more of the following: the load information of the target backup node and the execution capability information of the target backup node; the master node determines the target data range according to the load balancing algorithm based on the data range of the data to be scanned and the scanning information of the target backup node.

[0019] Therefore, during the first round of scanning, the master node allocates the data range to be scanned for each backup node according to the scanning information of the backup nodes and the load balancing algorithm. This can allocate the data range to be scanned by each backup node from a global perspective, so that the backup nodes with high processing efficiency can obtain more data range, scan more data, achieve better linear scheduling, and improve query performance.

[0020] In one possible implementation of the first aspect, the master node determines the target data range to be scanned by the target backup node based on the data range of the data to be scanned, including: the master node obtaining the scanning information of the target backup node; the master node determining that the second scanning operation has ended, and determining the target data range based on the data range of the data to be scanned and the data range of the target backup node corresponding to the second scanning operation, wherein the second scanning operation is the previous round of scanning operation of the first scanning operation.

[0021] Therefore, after performing the first round of operations, the master node can directly allocate the data range corresponding to the subsequent rounds of scanning operations based on the data range allocated in the previous round of scanning operations.

[0022] In one possible implementation of the first aspect, the master node is the coordinating node in the database system, and the backup node is the execution node in the database system.

[0023] Therefore, when executing a multi-machine parallel query plan, the master node acts as the coordinating node in the database system, and the standby node acts as the execution node in the database system.

[0024] A second aspect of this application provides a database system comprising a master node and at least one backup node, the at least one backup node including a target backup node. The database system is deployed in cloud computing resources managed by a cloud management platform. The master node is configured to: acquire query requests; generate a single-machine query plan and a multi-machine parallel query plan based on the query requests; evaluate the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan; and, if the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, send a first execution plan to the target backup node among the at least one backup node, wherein the multi-machine parallel query plan includes the first execution plan executed by the target backup node and a second execution plan executed by the master node. The master node and the at least one backup node are configured to execute the multi-machine parallel query plan to obtain query results corresponding to the query requests.

[0025] In one possible implementation of the second aspect, the master node is also used to: evaluate the query cost of a single-machine query plan and the query cost of a multi-machine parallel query plan based on a cost evaluation model; and compare the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan.

[0026] In one possible implementation of the second aspect, the query cost includes one or more of the following: execution startup cost, data access cost, data movement cost, or data aggregation cost, wherein the execution startup cost is used to indicate the startup cost of the query plan initialization thread and the query task allocation, the data access cost is used to indicate the data computation cost and the data storage cost, the data movement cost is used to indicate the data communication cost, and the data aggregation cost is used to indicate the cost of data execution aggregation to obtain the final result.

[0027] In one possible implementation of the second aspect, the master node is also used to execute the single-machine query plan to obtain the query result corresponding to the query request when the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan.

[0028] In one possible implementation of the second aspect, the master node is further configured to: obtain the status information of the target backup node, wherein the status information of the target backup node includes one or more of the following: the computing power information of the target backup node, the network transmission capability information of the target backup node, or the memory overhead information of the target backup node; and determine the amount of task to be performed by the target backup node in executing the first execution plan based on the status information of the target backup node.

[0029] In one possible implementation of the second aspect, the query request includes a first scan operation, which is used to indicate the data range of the data to be scanned. The master node is further used to determine the data range of the data to be scanned included in the first scan operation. The master node is further used to determine the target data range to be scanned by the target backup node based on the data range of the data to be scanned. The master node is further used to send the target data range to the target backup node. The target backup node is used to scan the data corresponding to the target data range.

[0030] In one possible implementation of the second aspect, the master node is further configured to: create and save the scanning information of the target backup node, wherein the scanning information of the target backup node includes one or more of the following: the load information of the target backup node, the execution capability information of the target backup node; and determine the target data range based on the data range of the data to be scanned and the scanning information of the target backup node, according to a load balancing algorithm.

[0031] In one possible implementation of the second aspect, the master node is further configured to: obtain scanning information of the target backup node; determine the end of the second scanning operation; and determine the target data range based on the data range of the data to be scanned and the data range of the target backup node corresponding to the second scanning operation, wherein the second scanning operation is the previous round of scanning operation of the first scanning operation.

[0032] In one possible implementation of the second aspect, the master node is the coordinating node in the database system, and the backup node is the execution node in the database system.

[0033] A third aspect of this application provides a computing device cluster, which includes at least one computing device, each computing device including a processor and a memory, wherein the memory of the at least one computing device stores computer-executable instructions that can run on the processor, and when the computer-executable instructions are executed by the processor, the processor executes a method as described in the first aspect or any possible implementation thereof.

[0034] The fourth aspect of this application provides a computer-readable storage medium for storing one or more computer-executable instructions, wherein when the computer-executable instructions are executed by a cluster of computing devices, the cluster of computing devices executes a method as described in the first aspect or any possible implementation thereof.

[0035] The fifth aspect of this application provides a computer program product storing one or more computer execution instructions, the computer program product including computer execution instructions, which, when executed by a cluster of computing devices, execute a method as described in the first aspect or any possible implementation thereof.

[0036] A sixth aspect of this application provides a chip system including a processor for supporting a cluster of computing devices to implement the functions described in the first aspect or any possible implementation thereof. In one possible design, the chip system may further include a memory for storing necessary program instructions and data. This chip system may be composed of chips or may include chips and other discrete devices.

[0037] The technical effects of the second to sixth aspects or any of their possible implementations can be found in the first aspect or the technical effects of its related possible implementations, and will not be repeated here. Attached Figure Description

[0038] Figure 1 is a structural schematic diagram of an implementation scenario of a database system provided in an embodiment of this application;

[0039] Figure 2 is a schematic diagram of the deployment of basic resources in a data center according to an embodiment of this application;

[0040] Figure 3 is a schematic diagram of the architecture of a database system provided in an embodiment of this application;

[0041] Figure 4 is a schematic diagram of the structure of a computing node provided in an embodiment of this application;

[0042] Figure 5 is a flowchart illustrating a data query method provided in an embodiment of this application;

[0043] Figure 6 is a schematic diagram of the structure of a database system provided in an embodiment of this application;

[0044] Figure 7 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0045] Figure 8 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0046] Figure 9 is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0047] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0048] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0049] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not expressly listed or inherent to those processes, methods, products, or apparatus.

[0050] The concepts involved in this disclosure are explained below:

[0051] Coordinating Node: Responsible for receiving access requests from applications, decomposing query tasks, scheduling tasks for parallel execution on various query execution nodes, and summarizing the results from all query execution nodes. Any node in the cluster that receives a query can become a query coordinating node, and a query coordinating node can also act as a query execution node. In this disclosure, the role of the coordinating node is a logical division.

[0052] Execution Node: Responsible for executing data query tasks and returning execution results to the query coordinating node. All nodes in the cluster can act as query execution nodes; in this disclosure, the role of execution nodes is a logical division.

[0053] Online transaction processing (OLTP), also known as transaction-oriented processing, is characterized by the ability of user data received at the front end to be immediately transmitted to the computing center for processing and the results to be provided in a very short time. It is one of the ways to respond quickly to user operations.

[0054] Online analytical processing (OLAP) is an analytics-oriented processing procedure that typically involves querying many, or even all, records in a database to retrieve large amounts of data for analysis, helping companies extract insights from their transaction data to make decisions using their reports.

[0055] A data dictionary is a database object that stores information about the database structure. It contains definition information for all objects in the database and manages the database's metadata to support database management.

[0056] In today's digital age, businesses and organizations are facing increasingly complex operations and more diverse database needs. Different business modules may have different access patterns and performance requirements. Traditional OLTP systems need to respond quickly to transaction processing and ensure data consistency and integrity. However, with the rapid development of information technology and the explosive growth of data volume, businesses and organizations need to store and manage massive amounts of data while also being able to quickly access and analyze it. Therefore, the OLAP capabilities of databases have become a crucial criterion for technology selection and performance evaluation.

[0057] In large-scale query and analysis scenarios, single-machine database systems often face resource constraints in areas such as central processing unit (CPU), memory, and input / output (I / O). For complex multi-table joins and aggregation calculations, the processing power of a single machine may not meet business needs, creating a single point of failure. To improve data processing speed and efficiency, multi-machine parallel computing technology has emerged. Multi-machine parallel computing typically involves breaking down a large computational task into multiple smaller subtasks, which are then distributed across different computing nodes for simultaneous processing. These nodes can be physical servers or virtual machines, connected via a network and working collaboratively to complete the entire computational task, achieving data parallelism and task parallelism, thereby shortening the execution time of the computational task. For scenarios involving large-scale data processing, complex queries, and high concurrency access, it can significantly improve execution performance. Furthermore, when one computing node fails, other computing nodes can continue processing the computational task, improving system stability and reliability.

[0058] Currently, existing distributed database deployments, based on multi-machine parallel computing technology, use a single compute node as the master node, responsible for generating and scheduling distributed execution plans. First, the compute node receives Structured Query Language (SQL) requests from the application layer or the management node, performs logical and physical optimizations, generates a distributed query plan that satisfies distributed transaction consistency, distributes the plans to various data nodes, and finally aggregates the computation results of the SQL requests. By distributing the tasks corresponding to the distributed query plan across multiple compute nodes, the data processing time is shortened, and query response speed is improved. However, using distributed databases requires application-side migration, and considering the selection of distribution keys for various business data tables presents challenges in business distributed transformation. Furthermore, in distributed scenarios, due to the randomness of the distribution of business data tables, storage skew and data skew issues can easily arise during production use due to changes in data distribution as business grows.

[0059] In centralized master-slave database deployments, existing parallel query technologies typically leverage the multi-threading capabilities of a single machine. During a query, data is sharded across different threads, with multiple threads performing parallel computations within a single compute node. The results are then pipelined and aggregated into a single thread. This is achieved through multi-CPU or multi-core CPU technology, also known as symmetrical multi-processing (SMP). SMP improves performance by sharing memory and using parallel operators within a single machine, fully utilizing the CPU, memory, and I / O capabilities of the system to complete queries within a single compute node. However, the execution capability of single-machine parallel queries is limited by the resource constraints of a single machine. A single complex SQL request cannot be executed in parallel across nodes. Data queries executed on the current compute node utilize that node's computing power, while the computing power of other compute nodes is not effectively utilized, leading to a waste of computing resources within the cluster and failing to achieve effective multi-machine parallelism.

[0060] Therefore, it is evident that in related technologies, centralized master-slave deployments of databases cannot effectively utilize multi-node hardware resources, and the performance and throughput of query plans need improvement. Furthermore, the parallel capabilities of both distributed and centralized master-slave deployments are limited. For complex query operations, transparency to users is not achieved, meaning users still need to make adaptations and modifications.

[0061] In view of this, embodiments of this application provide a data query method and a database system. The database system includes a master node and at least one backup node, the at least one backup node including a target backup node, and the database system is deployed in cloud computing resources managed by a cloud management platform. In this database system, the master node is used to obtain query requests and, based on the query requests, generate a single-machine query plan and a multi-machine parallel query plan, respectively. The master node is also used to evaluate the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan, and, if the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, send a first execution plan executed by the target backup node to the target backup node among the at least one backup node. The multi-machine parallel query plan includes the first execution plan executed by the target backup node and a second execution plan executed by the master node. The master node and at least one backup node are used to execute the multi-machine parallel query plan to obtain the query results corresponding to the query request.

[0062] In this way, multi-machine parallel optimization is achieved in a centralized master-slave database system. User query requests can be received by any compute node in the database system, which acts as the master node. The master node generates both a single-machine query plan and a multi-machine parallel query plan, evaluates the query costs of both plans, and executes the multi-machine parallel query plan only if its cost is less than or equal to the cost of the single-machine plan. This process is transparent to upper-level user applications and imperceptible to the user. Simultaneously, the master node and at least one standby node in the database system execute the multi-machine parallel query plan, enabling cross-node multi-machine parallel query execution and improving query performance.

[0063] This article provides a detailed introduction to the technical solution of this application from multiple perspectives, including implementation scenarios, methods and processes, hardware devices, and software devices.

[0064] The following are examples illustrating the implementation scenarios of the embodiments of this application.

[0065] Figure 1 is a structural diagram of an implementation scenario of a database system provided in this application embodiment. As shown in Figure 1, the implementation scenario includes: a data center 1 and a client 2. Data center 1 and client 2 can establish a communication connection through a network. Optionally, the network can be the Internet, or other networks; this application embodiment does not limit the specific network. Tenants can interact with data center 1 through client 2. For example, a tenant can send cloud service requests and other information to data center 1 through client 2. Data center 1 responds based on the information sent by client 2.

[0066] Data center 1 houses a large amount of infrastructure owned by a cloud service provider, such as computing resources, storage resources, and network resources. For example, computing resources can be computing devices (such as servers) capable of providing computing power. As shown in Figure 1, data center 1 includes a cloud management platform and infrastructure (not shown in Figure 1). The cloud management platform and infrastructure are connected via an internal data center network. The cloud management platform manages the infrastructure. The infrastructure provides public cloud services. The infrastructure includes multiple servers. Cloud services are optionally deployed on the servers. Cloud services are implemented by running virtual instances, and are therefore also referred to as virtual instances deployed on servers to implement tenant business. Tenants can send cloud service requests and related information to the server through their client 2. The server can process the cloud service requests and related information and provide cloud services to the tenant based on the processed cloud service requests and related information. For example, the server can query data through the database system provided in this embodiment.

[0067] The cloud management platform can be logically divided into: tenant console, compute management service, network management service, storage management service, authentication service, and image management service. The tenant console provides a user interface or application programming interface (API) for interaction with tenants. The compute management service manages servers running virtual instances and bare metal servers. The network management service manages network services (such as gateways and firewalls). The storage management service manages storage services (such as data bucket services). The authentication service manages tenant accounts and passwords. The image management service manages virtual instance images.

[0068] In the implementation scenario shown in Figure 1, a data center contains multiple servers. The servers consist of a hardware layer and a software layer. The hardware layer comprises the standard server configuration, including hardware devices such as processors, memory, network interface cards (NICs), disks, and buses. The software layer includes the operating system installed and running on the server. The operating system relative to the virtual machine can be called the host operating system. The host operating system runs a virtual machine manager (also known as a hypervisor). The virtual machine manager's role is to implement compute virtualization, network virtualization, and storage virtualization for the virtual machines, and to manage the virtual machines.

[0069] The virtual machine manager runs a cloud management platform client. This client receives control plane commands from the cloud management platform, creates virtual instances on the server based on these commands, and manages the virtual instances throughout their lifecycle. For example, the client can monitor the hardware resource usage of the server in real time and report it to the cloud management platform. When the cloud management platform confirms that a virtual instance needs to be created on a specific server, it sends a virtual instance creation command to the client on that server. Upon receiving the command, the client creates the virtual instance on that server. In this way, tenants can create, manage, log in to, and operate virtual instances within the data center through the cloud management platform.

[0070] Servers can run virtual machines of different specifications. Virtual machine specifications are categorized as: general-purpose computing, memory-optimized, ultra-large memory, etc., with specific specifications under each type. After a tenant selects a virtual machine specification, the cloud management platform selects a server in the data center that supports that specification and ensures sufficient idle hardware resources on that server. Then, it creates and configures the virtual machine with that specification on that server. Configuring servers through the cloud management platform allows for the analysis and planning of server hardware resources. Based on the server's hardware performance, it plans the corresponding computing products for the physical hardware, such as planning virtual machines of different specifications, to meet the diverse needs of different tenants. Furthermore, differentiated pricing strategies can be implemented based on the performance differences of virtual machines of different specifications. For example, high-performance virtual instances can be sold at a higher price, while ordinary performance virtual instances can be sold at a lower price, allowing tenants to purchase virtual instances as needed.

[0071] In one implementation, as shown in Figure 2, the location of basic resources in a data center can be described by cloud resource deployment regions (regions) and availability zones (AZs). Tenants can choose to deploy cloud services based on resources in specific regions and AZs. Regions are defined based on geographical location and network latency. Using the same resource pool within the same region can be understood as sharing public services such as elastic computing, block storage, object storage, virtual private cloud (VPC) networks, elastic internet protocol (EIP) addresses, and images. Regions are divided into general regions and dedicated regions. General regions refer to regions that provide general cloud services to public tenants. Dedicated regions refer to dedicated regions that host the same type of business or provide business services to specific tenants. A region typically includes multiple AZs. Multiple AZs within a region are connected via high-speed fiber optic cables to meet the needs of tenants building high-availability systems across AZs. An AZ is a collection of one or more data centers as shown in Figure 2. Computing, network, and storage resources within an AZ are logically divided into multiple clusters.

[0072] Tenants can send instructions to the cloud management platform through their client 2 to create, manage, log in to, and operate virtual instances on the server, and use the cloud services provided by these virtual instances. For example, the cloud management platform can provide an access interface. This interface can be provided either as a user interface or an API. Tenants can operate their client to remotely access the access interface to register a cloud account and password on the cloud management platform, and then log in using these accounts and passwords. The cloud management platform can also authenticate the cloud account and password. After successful authentication, the tenant can further select and purchase a virtual instance with specific specifications (processor, memory, disk) on the cloud management platform. After the tenant successfully purchases the virtual instance, the cloud management platform provides the tenant with a remote login account and password for the purchased virtual instance. The tenant can use the remote login account and password to remotely log in to the virtual instance on their client, install and run their application within the virtual instance, and use the application to implement their business operations.

[0073] Client 2 can be selected from computers, personal computers, laptops, mobile phones, smartphones, tablets, cloud servers, portable mobile terminals, multimedia players, e-book readers, wearable devices, smart home appliances, artificial intelligence devices, smart wearable devices, smart in-vehicle devices, or Internet of Things devices, etc.

[0074] In one implementation, the database system provided in this application embodiment can be implemented by running an executable program on a computing device in data center 1. Optionally, the database system provided in this application embodiment can be applied to a cloud service management system. The cloud service management system is deployed on a server managed by a cloud management platform. By running the executable program of the database system provided in this application embodiment, the cloud service management system can implement the data query method provided in this application embodiment. Furthermore, the executable program implementing the data query method and the database system can optionally be presented in the form of an application installation package. After the server installs the application installation package, it can implement the data query method and the database system provided in this application embodiment by running the executable program therein.

[0075] It should be understood that the above content is an exemplary description of the implementation scenarios of the database system provided in the embodiments of this application, and does not constitute a limitation on the implementation scenarios of the database system. As those skilled in the art will know, as business needs change, the implementation scenarios can be adjusted according to application requirements, and the embodiments of this application do not specifically limit them. Furthermore, when the database system provided in the embodiments of this application is applied to other scenarios, the executable program of the method can also be presented in the form of an application installation package or in other ways, and the embodiments of this application do not list them all.

[0076] The implementation process of the database system provided in this application embodiment will be illustrated below. Figure 3 is a schematic diagram of the architecture of a database system provided in this application embodiment. As shown in Figure 3, the database system includes infrastructure for providing computing and storage services. This infrastructure includes multiple computing nodes and storage engines. The multiple computing nodes are interconnected, with one computing node acting as the master node and the other computing nodes acting as backup nodes. The computing nodes access the data stored in the storage engine by calling the storage engine's interface. The master node, backup nodes, and storage engine will be described below.

[0077] The primary node is presented to the user's client. When a user's client sends a query request, it can connect to any of the multiple computing nodes contained in the database system and send the query request to that computing node so that the computing node can complete the corresponding data processing for the query request. Other computing nodes are presented as backup nodes, and the primary node is deployed in a one-primary-multiple-backup configuration.

[0078] It is worth noting that, as shown in Figure 4 (Figure 4 is a schematic diagram of the structure of a computing node provided in an embodiment of this application), for the master node and any standby node, the computing node includes a parser, a preprocessor, an optimizer, and an executor. The parser receives query requests, which can be understood as a query statement. The parser generates a corresponding parse tree based on the query statement and, based on syntax analysis, determines whether the keywords in the query statement satisfy SQL syntax. For example, it verifies whether incorrect keywords are used or whether the order of the keywords is correct. For instance, the query statement could be "select * from UserInfo where name = 'tml' and age = '26' and sex = 'Male'". The parser extracts the keywords: select, from, where, and and from the query statement and determines whether these keywords satisfy SQL syntax. The preprocessor further checks the validity of the parse tree based on SQL rules, such as checking if the data table and columns exist, and whether there is any ambiguity in the parsed column names and aliases. Next, the preprocessor verifies user permissions to check if the user has the corresponding operation privileges. Using the above query as an example, the preprocessor further obtains the UserInfo table name and column names: name, age, and sex, and checks if these elements all exist. If they all exist, permissions are verified. The optimizer executes the query according to what the SQL engine considers the optimal solution. For example, when there are multiple indexes on the table, it decides which index to use; or when a statement involves multiple table joins, it determines the join order of the tables. The optimizer converts the query into an execution plan. A query can have many execution methods, all returning the same result. Finally, it finds the optimal execution plan. Again using the above query as an example, the optimizer determines which column to retrieve first, generating various plans [name->age->sex, name->sex->age, age->sex->name, etc.], and ultimately selects the optimal and least costly plan for execution. The executor executes the SQL statement step by step according to the instructions given in the execution plan. When execution begins, it first checks whether the user has permission to execute a query on the table. If not, it returns a permission error. If the user has permission, it continues execution and returns the final result to the client.

[0079] The storage engine, consisting of multiple storage nodes, is responsible for storing and retrieving data.

[0080] Furthermore, for any one of the multiple computing nodes and any one of the multiple storage nodes, since the database system can be built based on cloud technology, the computing node or storage node can be a cloud instance in m. The cloud instance can be presented in various ways. For example, the cloud instance can be a physical server specified by the cloud management platform, or a virtual machine (VM) created by the cloud management platform on the physical server using virtualization technology, or a container (Docker) created by the cloud management platform on the physical server using virtualization technology, or a micro VM created by the cloud management platform on the physical server using virtualization technology, or a container created by the cloud management platform on the physical server using virtualization technology, or a bare metal server specified by the cloud management platform, etc.

[0081] Furthermore, multiple compute nodes and multiple storage nodes can typically be set up in one or more sites. Sites can be presented in various forms. For example, one or more sites can be multiple regions in the infrastructure, or multiple availability zones in the infrastructure, or at least one edge site and at least one central cloud site in the infrastructure, and so on.

[0082] To better understand the workflow of the aforementioned database system, the following description, in conjunction with Figure 5, further illustrates this workflow. Figure 5 is a flowchart illustrating a data query method provided in an embodiment of this application. As shown in Figure 5, this method is applied to the database system shown in Figure 3, which includes a master node and at least one backup node. The method includes steps 501 to 505.

[0083] Step 501: The master node obtains the query request.

[0084] In this embodiment, when a user has pending requests, the user can send a query request to any one of the multiple computing nodes through their client. The computing node that receives the query request is the master node. The query request contains a query statement, which can be used to instruct the master node to process the pending data. Therefore, the query request can be used to request the master node to process the pending data that the user needs to process.

[0085] Step 502: Based on the query request, the master node generates a single-machine query plan and a multi-machine parallel query plan respectively.

[0086] After receiving the query request, the master node generates a single-machine query plan and a multi-machine parallel query plan based on the query request, which are used to make subsequent decisions on whether to adopt the multi-machine parallel query plan.

[0087] It is worth noting that a single-machine query plan can include both single-machine query plans and single-machine parallel query plans. This application does not limit the type of single-machine query plan.

[0088] Step 503: The master node evaluates the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan, respectively.

[0089] The master node evaluates the query cost of a single-machine query plan and the query cost of a multi-machine parallel query plan. This query cost can be evaluated using a cost evaluation model to obtain the query cost of both the single-machine and multi-machine parallel query plans.

[0090] It should be noted that query cost assessment can be based on collected statistical information to establish cost assessment models for different query plans, thereby achieving a comprehensive cost assessment of the execution efficiency and execution cost (such as time cost and the cost of computing resources consumed) of the query plan. This allows for the calculation of the comprehensive execution cost of each query plan, enabling the selection of the query plan with the optimal execution efficiency and execution cost from multiple query plans as the target query plan.

[0091] In some embodiments, the statistical information includes table-level statistics and column-level statistics. The table-level statistics may include the number of tuples and the number of pages occupied by the table. The column-level statistics may include the width of the attribute, the maximum and minimum values ​​of the attribute, and high-frequency values.

[0092] Below, we can provide an example of how to evaluate the cost of a multi-machine parallel query plan in the cost evaluation model.

[0093] In one embodiment, the query cost of the multi-machine parallel query plan includes at least one of the following cost evaluation metrics:

[0094] The costs include startup cost, data access cost, data movement cost, and data aggregation cost.

[0095] The following section provides explanations for each cost assessment indicator, along with specific examples.

[0096] (1) Execution startup cost

[0097] The execution startup cost indicates the startup cost of initializing the query plan threads and allocating query tasks. This cost includes the startup cost of initializing the multi-machine parallel query plan threads, query task sharding, and network communication. This execution startup cost can be represented by `MULTI_PLAN_STARTUP_COST`.

[0098] (2) Data access cost

[0099] Data access cost, used to indicate the cost of data computation and the cost of data storage, is calculated from the following items:

[0100] Where n is the number of compute nodes participating in processing the query request, NODE_GROUP_TUPLES is the number of data rows allocated to each compute node for access, and CPU_PER_TUPLE_COST and IO_PER_TUPLE_COST are the CPU cost and IO cost, respectively.

[0101] (3) Data migration cost

[0102] Data movement cost, used to indicate the cost of data communication, is calculated from the following:

[0103] Where n is the number of compute nodes involved in processing the query request, RETURN_TUPLES is the number of data rows returned by each compute node after computation, and NETWORK_COST is the overhead of data transmission between compute nodes, including network overhead and the overhead of data serialization and / or data deserialization.

[0104] (4) Cost of data aggregation

[0105] Data aggregation cost is used to indicate the cost of performing data aggregation to obtain the final result. Data aggregation overhead includes the overhead of the master node performing data aggregation, sorting and other operations, which can be represented by MULTI_PLAN_GATHER_COST.

[0106] In this way, given a fixed amount of computing resources, the query cost of a multi-machine parallel query plan can be calculated through cost evaluation. The query cost of the multi-machine parallel query plan can be represented as TOTAL_COST(MULTI_PLAN), and can be calculated using the following methods:

[0107] In some embodiments, compared with the query cost of a multi-machine parallel query plan, the query cost of a single-machine query plan does not include the data communication cost in the data movement cost, that is, it does not include the overhead of data transmission between computing nodes.

[0108] After calculating the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan, the master node compares the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan to determine the actual query plan to be executed.

[0109] Step 504: If the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the master node sends a first execution plan to the target backup node among at least one backup node. The multi-machine parallel query plan includes the first execution plan executed by the target backup node and the second execution plan executed by the master node.

[0110] In this embodiment, when the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the master node divides the multi-machine parallel query plan into multiple sub-plans. Each sub-plan is executed by a backup node, and the backup nodes synchronize data during the parallel execution of the sub-plans through synchronization operators. Specifically, after dividing the multi-machine parallel query plan into multiple sub-plans, a target backup node is determined to execute the first execution plan. After the master node executes the second execution plan, the master node sends the first execution plan executed by the target backup node to at least one of the backup nodes. The multi-machine parallel query plan includes the first execution plan executed by the target backup node and the second execution plan executed by the master node.

[0111] If the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan, the master node will execute the single-machine query plan to obtain the query result corresponding to the query request.

[0112] In this way, after generating single-machine query plans and multi-machine parallel query plans, the master node evaluates the query costs of the single-machine query plan and the multi-machine parallel query plan respectively. It compares the query costs of the single-machine query plan and the multi-machine parallel query plan and determines whether to execute the multi-machine parallel query plan or the single-machine query plan if the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, and if the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan. This achieves transparency to upper-layer user business and makes decisions on whether to execute the multi-machine parallel query plan or the single-machine query plan without the user's awareness.

[0113] Step 505: The master node and at least one backup node execute a multi-machine parallel query plan to obtain the query results corresponding to the query request.

[0114] If the primary node determines that the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the primary node and at least one backup node jointly execute the multi-machine parallel query plan to obtain the query result corresponding to the query request.

[0115] In this way, when executing a multi-machine parallel query plan, it is possible to execute the query plan across nodes, giving full play to the advantages of multi-node parallel execution and improving query performance.

[0116] Furthermore, in this embodiment, during the process of the master node generating the multi-machine parallel query plan, the master node transforms and rewrites the query tree to eliminate redundant operations and reduce data transmission. In addition, the master node obtains the status information of the target backup nodes, which includes one or more of the following: the target backup node's computing power information, the target backup node's network transmission capability information, or the target backup node's memory overhead information. Based on the target backup node's status information, the master node determines the workload of the target backup node executing the first execution plan. In this way, the master node can select a suitable physical execution method for each backup node executing the multi-machine parallel query plan. It is worth noting that the workload of the first execution plan can be the workload of the target backup node executing the first execution plan for the first time.

[0117] Furthermore, in this embodiment of the application, the process of generating a multi-machine parallel query plan by the master node also includes the data distribution method among the backup nodes, the execution steps of the query plan, and the aggregation method.

[0118] Furthermore, the query request includes a scan operation, which indicates the range of data to be scanned. Specifically, during the execution of the multi-machine parallel query plan by the primary node and the at least one backup node, a scan operator needs to be executed. After the primary node sends the corresponding execution plan to the at least one backup node, it determines the range of data to be scanned for each backup node based on the data range of the data to be scanned, and then sends the data range corresponding to each backup node to the corresponding backup node so that the backup node can execute the corresponding scan operation.

[0119] In some examples, when a target standby node performs its first scan operation, it reports its load information and execution capability information to the master node. The master node then creates scan information for the target standby node based on this information. This scan information includes the target standby node's load and execution capability information. This first scan operation is the initial scan operation. After creating the scan information, the master node determines the target data range to be scanned by the target standby node based on the data range of the data to be scanned and the target standby node's scan information, according to a load balancing algorithm. In this way, during the initial scan operation, the master node allocates the data range to be scanned for each standby node according to the standby node's scan information and the load balancing algorithm. This allows for a global allocation of the data range for each standby node, enabling high-efficiency standby nodes to acquire a larger data range and scan more data, thereby achieving better linear scheduling and improving query performance.

[0120] In some examples, after the target backup node performs the first round of scanning, the master node obtains the scanning information of the target backup node and further determines whether the second scanning operation has ended. If the second scanning operation is confirmed to be complete, the master node determines the target data range of the target backup node for the first scanning operation based on the data range of the data to be scanned and the data range of the target backup node corresponding to the second scanning operation. Here, the second scanning operation is the previous round of scanning operation preceding the first scanning operation. In this way, after performing the first round of operations, the master node can directly allocate the data range corresponding to subsequent rounds of scanning operations based on the data range allocated in the previous round of scanning operations.

[0121] Furthermore, when executing a multi-machine parallel query plan, the master node acts as the coordinating node in the database system, and the standby nodes act as the execution nodes in the database system.

[0122] In this embodiment, for a query request sent by a user's client, the master node generates both a single-machine query plan and a multi-machine parallel query plan based on the query request. It then evaluates the query cost of both the single-machine and multi-machine parallel query plans and compares them. If the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the master node and at least one backup node execute the multi-machine parallel query plan to obtain the query result corresponding to the query request. If the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan, the master node executes the single-machine query plan to obtain the query result corresponding to the query request. Therefore, in a centralized database system deployment scenario, the master node decides whether to execute a multi-machine parallel query plan or a single-machine query plan based on the query cost, without requiring any adaptation modifications from the user. This achieves transparency to upper-layer user services and leverages the advantages of multi-node parallel execution by executing the multi-machine parallel query plan across nodes, thereby improving query performance.

[0123] The above is a detailed description of the data query method provided in the embodiments of this application. The database system provided in the embodiments of this application will be described below. Figure 6 is a schematic diagram of the structure of a database system provided in an embodiment of this application. As shown in Figure 6, the database system includes a master node and at least one backup node. Figure 6 uses a target backup node among the at least one backup node as an example. The database system includes:

[0124] Master node 601 is used to receive query requests;

[0125] The master node 601 is also used to generate single-machine query plans and multi-machine parallel query plans based on query requests.

[0126] The master node 601 is also used to evaluate the query cost of a single-machine query plan and the query cost of a multi-machine parallel query plan, respectively.

[0127] The master node 601 is also used to send a first execution plan executed by the target backup node to at least one backup node when the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan. The multi-machine parallel query plan includes the first execution plan executed by the target backup node and the second execution plan executed by the master node.

[0128] The primary node 601 and at least one backup node are used to execute a multi-machine parallel query plan to obtain the query results corresponding to the query request.

[0129] In one possible implementation, the master node 601 is also used to: evaluate the query cost of a single-machine query plan and the query cost of a multi-machine parallel query plan based on a cost evaluation model; and compare the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan.

[0130] In one possible implementation, the query cost includes one or more of the following: execution startup cost, data access cost, data movement cost, or data aggregation cost, wherein the execution startup cost is used to indicate the startup cost of the query plan initialization thread and the query task allocation, the data access cost is used to indicate the data computation cost and the data storage cost, the data movement cost is used to indicate the data communication cost, and the data aggregation cost is used to indicate the cost of data execution aggregation to obtain the final result.

[0131] In one possible implementation, the master node 601 is also used to execute a single-machine query plan to obtain the query result corresponding to the query request when the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan.

[0132] In one possible implementation, the master node 601 is further configured to: obtain the status information of the target backup node, wherein the status information of the target backup node includes one or more of the following: the computing power information of the target backup node, the network transmission capability information of the target backup node, or the memory overhead information of the target backup node; and determine the amount of task to be performed by the target backup node in executing the first execution plan based on the status information of the target backup node.

[0133] In one possible implementation, the query request includes a first scan operation, which is used to indicate the data range of the data to be scanned. The master node 601 is also used to determine the data range of the data to be scanned included in the first scan operation. Based on the data range of the data to be scanned, the target data range to be scanned by the target backup node is determined, and the target data range is sent to the target backup node. The target backup node 602 is used to scan the data corresponding to the target data range.

[0134] In one possible implementation, the master node 601 is further configured to: create and save the scan information of the target backup node, wherein the scan information of the target backup node includes one or more of the following: the load information of the target backup node, the execution capability information of the target backup node; and determine the target data range based on the data range of the data to be scanned and the scan information of the target backup node, according to a load balancing algorithm.

[0135] In one possible implementation, the master node 601 is further configured to: obtain scanning information of the target backup node; determine the end of the second scanning operation; and determine the target data range based on the data range of the data to be scanned and the data range of the target backup node corresponding to the second scanning operation, wherein the second scanning operation is the previous round of scanning operation of the first scanning operation.

[0136] In one possible implementation, the master node is the coordinating node in the database system, and the backup node is the execution node in the database system.

[0137] It should be noted that the information interaction and implementation process between the master node and / or the target backup node mentioned above are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in the embodiments of this application, and will not be repeated here.

[0138] The following provides examples illustrating the basic hardware structures involved in the embodiments of this application.

[0139] This application also provides a computing device 700. As shown in Figure 7, the computing device 700 includes a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, the memory 706, and the communication interface 708 communicate with each other via the bus 702. The computing device 700 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 700.

[0140] Bus 702 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 7, but this does not imply that there is only one bus or one type of bus. Bus 84 can include pathways for transmitting information between various components of computing device 700 (e.g., memory 706, processor 704, communication interface 708).

[0141] Processor 704 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0142] The memory 706 may include volatile memory, such as random access memory (RAM). The processor 704 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0143] The memory 706 stores executable program code, and the processor 704 executes the executable program code to implement the functions of the aforementioned master node and target standby node, thereby realizing the data query method applied to the computing device cluster in the above embodiment. That is, the memory 706 stores instructions for executing the data query method applied to the computing device cluster in the above embodiment.

[0144] The communication interface 708 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 700 and other devices or communication networks.

[0145] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0146] As shown in Figure 8, the computing device cluster includes at least one computing device 700. The memory 706 of one or more computing devices 700 in the computing device cluster may store the same instructions for executing data query methods.

[0147] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the data query method. In other words, a combination of one or more computing devices 700 can jointly execute the instructions for executing the data query method.

[0148] It should be noted that the memory 706 in different computing devices 700 within the computing device cluster can store different instructions, each used to execute a portion of the data query method's functionality. That is, the instructions stored in the memory 706 of different computing devices 700 can implement the functionality of one or more modules in the master node and the target standby node.

[0149] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 9 illustrates one possible implementation. As shown in Figure 9, two computing devices 700A and 700B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 706 in computing device 700A can store instructions for performing the functions of the master node. Simultaneously, the memory 706 in computing device 700B can store instructions for performing the functions of the target standby node. Alternatively, the memory 706 in computing device 700A can store instructions for performing some of the functions of the master node. Simultaneously, the memory 706 in computing device 700B can store instructions for performing another part of the functions of the master node and instructions for the target standby node.

[0150] It should be understood that the functions of computing device 700A shown in Figure 9 can also be performed by multiple computing devices 700. Similarly, the functions of computing device 700B can also be performed by multiple computing devices 700.

[0151] This application embodiment also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster in Figures 8 and 9. The difference is that the memory 706 in one or more computing devices 700 in this computing device cluster can store the same instructions for executing data query methods.

[0152] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the data query method. In other words, a combination of one or more computing devices 700 can jointly execute the instructions for executing the data query method.

[0153] It should be noted that the memory 706 in different computing devices 700 within the computing device cluster can store different instructions for executing parts of the data query method. That is, the instructions stored in the memory 706 of different computing devices 700 can implement the functions of one or more modules in the master node and the target standby node.

[0154] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute a data query method.

[0155] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute a data query method.

[0156] This application also provides a chip system including a processor for implementing the steps performed by the aforementioned computing device cluster. In one possible design, the chip system may further include a memory for storing necessary program instructions and data. This chip system may be composed of chips or may include chips and other discrete devices.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0159] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0161] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data query method, characterized in that, The method is applied to a database system, which includes a primary node and at least one secondary node, wherein the at least one secondary node includes a target secondary node. The database system is deployed in cloud computing resources managed by a cloud management platform. The method includes: The master node obtains the query request; Based on the query request, the master node generates a single-machine query plan and a multi-machine parallel query plan, respectively. The master node evaluates the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan, respectively. When the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, the master node sends a first execution plan to the target backup node among the at least one backup node, wherein the multi-machine parallel query plan includes the first execution plan executed by the target backup node and the second execution plan executed by the master node. The master node and the at least one backup node execute the multi-machine parallel query plan to obtain the query result corresponding to the query request.

2. The method according to claim 1, characterized in that, The master node evaluates the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan, including: The master node evaluates the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan based on the cost evaluation model. The master node compares the query cost of the single-machine query plan with the query cost of the multi-machine parallel query plan.

3. The method according to claim 1 or 2, characterized in that, The query cost includes one or more of the following: execution startup cost, data access cost, data movement cost, or data aggregation cost, wherein the execution startup cost is used to indicate the startup cost of the query plan initialization thread and the query task allocation, the data access cost is used to indicate the data computation cost and the data storage cost, the data movement cost is used to indicate the data communication cost, and the data aggregation cost is used to indicate the cost of data execution aggregation to obtain the final result.

4. The method according to claim 1, characterized in that, The method further includes: If the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan, the master node executes the single-machine query plan to obtain the query result corresponding to the query request.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The master node obtains the status information of the target backup node, wherein the status information of the target backup node includes one or more of the following: the computing power information of the target backup node, the network transmission capability information of the target backup node, or the memory overhead information of the target backup node. The master node determines the amount of work to be done by the target backup node in executing the first execution plan based on the status information of the target backup node.

6. The method according to any one of claims 1 to 5, characterized in that, The query request includes a first scan operation, which indicates the data range of the data to be scanned. The primary node and the at least one backup node execute the multi-machine parallel query plan, including: The master node determines the data range of the data to be scanned included in the first scan operation; The master node determines the target data range to be scanned by the target backup node based on the data range of the data to be scanned. The master node sends the target data range to the target backup node; The target backup node scans the data corresponding to the target data range.

7. The method according to claim 6, characterized in that, The master node determines the target data range to be scanned by the target backup node based on the data range of the data to be scanned, including: The master node creates and saves the scanning information of the target backup node, wherein the scanning information of the target backup node includes one or more of the following: the load information of the target backup node and the execution capability information of the target backup node. The master node determines the target data range based on the data range of the data to be scanned and the scanning information of the target backup node, according to a load balancing algorithm.

8. The method according to claim 6, characterized in that, The master node determines the target data range to be scanned by the target backup node based on the data range of the data to be scanned, including: The master node acquires the scan information of the target backup node; The master node determines that the second scanning operation has ended, and determines the target data range based on the data range of the data to be scanned and the data range of the target backup node corresponding to the second scanning operation, wherein the second scanning operation is the previous round of scanning operation of the first scanning operation.

9. The method according to any one of claims 1 to 8, characterized in that, The master node is the coordinating node in the database system, and the backup node is the execution node in the database system.

10. A database system, characterized in that, The database system includes a master node and at least one backup node, wherein the at least one backup node includes a target backup node. The database system is deployed in cloud computing resources managed by a cloud management platform. The master node is used to obtain query requests; The master node is also used to generate a single-machine query plan and a multi-machine parallel query plan based on the query request. The master node is also used to evaluate the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan, respectively. The master node is further configured to, when the query cost of the multi-machine parallel query plan is less than or equal to the query cost of the single-machine query plan, send a first execution plan executed by the target backup node to the target backup node among the at least one backup node, wherein the multi-machine parallel query plan includes the first execution plan executed by the target backup node and the second execution plan executed by the master node; The master node and the at least one backup node are used to execute the multi-machine parallel query plan to obtain the query result corresponding to the query request.

11. The database system according to claim 10, characterized in that, The master node is also used for: Based on the cost evaluation model, the query cost of the single-machine query plan and the query cost of the multi-machine parallel query plan are evaluated respectively. Compare the query cost of the single-machine query plan with the query cost of the multi-machine parallel query plan.

12. The database system according to claim 10 or 11, characterized in that, The query cost includes one or more of the following: execution startup cost, data access cost, data movement cost, or data aggregation cost, wherein the execution startup cost is used to indicate the startup cost of the query plan initialization thread and the query task allocation, the data access cost is used to indicate the data computation cost and the data storage cost, the data movement cost is used to indicate the data communication cost, and the data aggregation cost is used to indicate the cost of data execution aggregation to obtain the final result.

13. The database system according to claim 10, characterized in that, The master node is also used to execute a single-machine query plan to obtain the query result corresponding to the query request when the query cost of the multi-machine parallel query plan is greater than the query cost of the single-machine query plan.

14. The database system according to any one of claims 10 to 13, characterized in that, The master node is also used for: Obtain the status information of the target backup node, wherein the status information of the target backup node includes one or more of the following: the computing power information of the target backup node, the network transmission capability information of the target backup node, or the memory overhead information of the target backup node; Based on the status information of the target backup node, the amount of task to be performed by the target backup node in executing the first execution plan is determined.

15. The database system according to any one of claims 10 to 14, characterized in that, The query request includes a first scan operation, which indicates the data range of the data to be scanned. The master node is also used to determine the data range of the data to be scanned included in the first scanning operation; The master node is also used to determine the target data range to be scanned by the target backup node based on the data range of the data to be scanned. The master node is also used to send the target data range to the target backup node; The target backup node is used to scan the data corresponding to the target data range.

16. The database system according to claim 15, characterized in that, The master node is also used for: Create and save the scan information of the target backup node, wherein the scan information of the target backup node includes one or more of the following: the load information of the target backup node and the execution capability information of the target backup node; Based on the data range of the data to be scanned and the scanning information of the target backup node, the target data range is determined according to the load balancing algorithm.

17. The database system according to claim 15, characterized in that, The master node is also used for: Obtain the scanning information of the target backup node; Once the second scanning operation is determined to be complete, the target data range is determined based on the data range of the data to be scanned and the data range of the target backup node corresponding to the second scanning operation, wherein the second scanning operation is the previous round of scanning operation of the first scanning operation.

18. The database system according to any one of claims 10 to 17, characterized in that, The master node is the coordinating node in the database system, and the backup node is the execution node in the database system.

19. A computing device cluster, characterized in that, It includes at least one computing device, said at least one computing device including a processor and a memory; The processor is configured to execute instructions stored in the memory to cause the computing device cluster to perform the method as described in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a processor, causes the processor to perform the method as described in any one of claims 1 to 9.

21. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1 to 9.

Images