Data query method, device, database system and storage medium

Abstract

Embodiments of the present disclosure provide a data query method, a device, a database system and a storage medium. An improved query scheme is provided under a multi-node parallel query architecture. When sub-tasks are distributed to a plurality of parallel execution nodes, the execution nodes can respectively determine target partitions, in a target partition table, allocated to the sub-tasks, so that partition information in the target partition table can be transparently transmitted to the execution nodes. On this basis, in the execution nodes, the obtained partition information can be fully utilized to carry out intra-node scheduling. Therefore, scheduling processes inside the execution nodes no longer need to depend on a shuffle (re-distribution) mechanism, so that the performance overhead in intra-node scheduling processes can be effectively reduced, thereby improving the query execution efficiency in a multi-node parallel query scenario.

Description

Data query method, device, database system and storage medium

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on December 22, 2023, with application number 202311786596.8 and application name “A Data Query Method, Device, Database System and Storage Medium”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of database technology, and in particular to a data query method, device, database system, and storage medium. Background Art

[0003] Partitioning is a database optimization technique that divides a large logical table into multiple smaller physical tables according to specific rules, thereby improving query and maintenance efficiency. During the partitioning process, the database distributes data to different partitions according to the partitioning rules. Indexes and other optimization techniques can be used within the partitions to improve query efficiency. The large logical table is the partitioned table, and the smaller physical tables are partitions. Each partition independently organizes and manages data and indexes on the storage engine. When performing a join query, if both tables are partitioned and the partition key is the same or related to the join key, a partition-wise join can be achieved.

[0004] Currently, partition-wise join technology has been integrated with the multi-machine parallel query architecture, which divides and schedules query tasks based on partition information. This allows multiple nodes to independently handle the join work on their assigned partitions, thereby reducing unnecessary data transmission power consumption between nodes.

[0005] However, a single node in a multi-machine parallel query architecture typically launches multiple worker processes to parallelize the node's assigned subtasks. Within a single node, the fetched partitions are reduced to common data tables, requiring the node to rely on a shuffle (redistribution) mechanism for multi-threaded scheduling. As we all know, the shuffle mechanism has significant performance overhead, resulting in poor query execution efficiency within the node.

[0006] Summary of the Invention

[0007] Various aspects of the present disclosure provide a data query method, device, database system, and storage medium to improve query execution efficiency in a multi-machine parallel query scenario.

[0008] The present disclosure provides a query method adapted to any execution node among a plurality of execution nodes for parallel processing of a query task, wherein the query object of the query task is a target partition table. The method includes:

[0009] After receiving the subtasks split from the query task, determining the target partition to which the subtasks are assigned in the target partition table;

[0010] Configuring a scheduling relationship between the target partition and each work process corresponding to the subtask;

[0011] After the data corresponding to the target partition is pulled, the pulled data is scheduled to the work process associated with the partition to which it belongs based on the scheduling relationship, so that the work process can execute the query.

[0012] The embodiment of the present disclosure further provides an execution node, including a memory, a processor, and a communication component;

[0013] The memory is used to store one or more computer instructions;

[0014] The processor is coupled to the memory and the communication component, and is configured to execute the one or more computer instructions to perform the aforementioned query method.

[0015] An embodiment of the present disclosure also provides a database system, including multiple execution nodes for processing query tasks in parallel, wherein the multiple execution nodes respectively receive subtasks split from the query task, and a single execution node starts multiple parallel work processes for the subtasks it receives, and the single execution node is used to execute the aforementioned query method.

[0016] The embodiment of the present disclosure further provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes the one or more processors to execute the aforementioned query method.

[0017] In the embodiment of the present disclosure, an improved query scheme is proposed under the multi-machine parallel query architecture. After distributing subtasks to multiple parallel execution nodes, each execution node can respectively determine the target partition to which the subtask is assigned in the target partition table. In this way, the partition information in the target partition table can be transmitted to each execution node. On this basis, within the execution node, the scheduling relationship between the assigned partition and each work process started for the assigned subtask within the execution node can be configured, and the pulled data can be scheduled to the corresponding work process based on the scheduling relationship. That is, within the execution node, the acquired partition information can be fully utilized to perform scheduling within the node. This makes it possible for the scheduling process within the execution node no longer need to rely on the shuffle (redistribution) mechanism, which can effectively save the performance overhead of the scheduling process within the node, thereby improving the query execution efficiency in the multi-machine parallel query scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0019] FIG1 is a schematic diagram of the structure of a database system provided by an exemplary embodiment of the present disclosure;

[0020] FIG2 shows a logical diagram of an exemplary scheme for inter-node scheduling;

[0021] FIG3 is a schematic diagram of the internal logic structure of an execution node provided by an exemplary embodiment of the present disclosure;

[0022] FIG4 is a logic diagram of an implementation scheme for configuring a scheduling relationship provided by an exemplary embodiment of the present disclosure;

[0023] FIG5 is a logic diagram of an improved query solution provided by an exemplary embodiment of the present disclosure;

[0024] FIG6 is a flow chart of a query method provided by another exemplary embodiment of the present disclosure;

[0025] FIG7 is a schematic structural diagram of an execution node provided by yet another exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present disclosure.

[0027] Before describing in detail the technical solutions provided by the various embodiments of the present disclosure, several technical concepts involved in the present disclosure are briefly explained as follows.

[0028] Partitioning is a database optimization technique that divides large tables into multiple smaller tables according to specific rules, thereby improving query and maintenance efficiency. During the partitioning process, the database distributes data to different partitions according to the partitioning rules and uses indexes and other optimization techniques within the partitions to improve query efficiency.

[0029] Partitioned tables: Partitions can be implemented in a database using partitioned tables. A partitioned table splits a large table into multiple smaller tables, each called a partition. A partitioned table contains multiple partitions, each corresponding to an independent physical storage space.

[0030] Partition-wise joins: Partition-wise joins break down joins between partitioned tables into joins between matching partitions based on the join condition (join key). Partition-wise joins can be applied when both tables being joined are partitioned on the join key.

[0031] Multi-machine parallel query: This can be understood as multiple execution nodes processing the same query task in parallel. In practice, the query task can be split into multiple subtasks and assigned to multiple execution nodes. Each execution node can execute its assigned subtasks in parallel, and the output results of the multiple subtasks can be aggregated to produce the query task's output. A single execution node can provide multiple worker processes to process the assigned subtasks in parallel.

[0032] Shuffle (redistribution): This can be simply understood as calculating a hash value for each data record in a data table. The hash value then determines which node or process to dispatch the data to. When a query involves multiple tables, a full shuffle is performed on all tables. The hash join operator also involves a shuffle. The general logic is: a full hash value is calculated for each of the two tables to be hash joined. For each data record in one table, a full table probe is performed in the other table to determine whether the join condition is met. As is well known, shuffles often have significant performance overhead.

[0033] As described in the background technology section, current partition-wise joins primarily utilize partition information when scheduling tasks across multiple execution nodes. However, scheduling within a single execution node requires a shuffle (redistribution) mechanism, resulting in excessive performance overhead within the execution node and poor query execution efficiency.

[0034] To this end, the present disclosure proposes an improved query solution under a multi-machine parallel query architecture to optimize intra-node scheduling and thereby improve query execution efficiency.

[0035] The technical solutions provided by various embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0036] Figure 1 is a schematic diagram of the structure of a database system provided by an exemplary embodiment of the present disclosure. As shown in Figure 1, the database system includes multiple execution nodes. In this embodiment, the database system may adopt a storage and computing separation (storage and computing separation) architecture, for example, databases such as PolarDB. Referring to Figure 1, the database system also includes multiple storage nodes to form a storage layer; and the execution nodes in this embodiment can be understood as computing nodes, forming a computing layer. A database engine can be run in each execution node to be responsible for computing work; the data table is stored in the storage layer, and the execution node can pull the required data from the storage layer according to computing requirements. Here, no further explanation is given on the storage and computing separation architecture.

[0037] In this embodiment, a multi-machine parallel query architecture is used in the database system to provide query services. As explained in the previous article on the concept of multi-machine parallel query, after the user initiates a query task, the query task can be split into multiple subtasks and assigned to multiple execution nodes. In addition, in this embodiment, the data table executed by the query task is a partitioned table. In this way, the process of allocating subtasks among multiple execution nodes can be implemented according to the scheduling mechanism provided in the aforementioned partition-wise join technology. In this embodiment, this scheduling process is not limited. Figure 2 shows a logical schematic diagram of an exemplary scheme for inter-node scheduling. Referring to Figure 2, in this exemplary solution, based on the logical structure of the database system, after a user initiates a query statement, the query optimizer can parse the query statement. The database scheduler then generates a physical execution plan (as a query task) for the query statement based on the parsed results, and splits the physical execution plan into multiple sharding plans (as subtasks). Each sharding plan has consistent task logic, but processes different partitions in the target partition table specified by the query statement. The scheduler can then distribute the sharding plans to multiple executors, using scheduling rules such as load balancing, which are not specifically exemplified here. The executors are provided by the execution nodes in this embodiment, so that the multiple subtasks split from a query task are assigned to the execution nodes.

[0038] In existing partition-wise join (PartitionWise Join) technology, from the perspective of a single execution node, upon receiving a subtask, it begins to process it as a normal query task. The execution node is no longer aware of the partition information of the target partition table and instead relies on the shuffle mechanism for intra-node scheduling.

[0039] This embodiment abandons the intra-node scheduling method that relies on the shuffle mechanism and proposes a new intra-node scheduling concept: partition information of the partition table is transparently transmitted to the execution node. The execution node then performs intra-node scheduling based on the partition information, without relying on the shuffle mechanism. This concept is explained in detail below.

[0040] For ease of description, this embodiment will illustrate the concept from the perspective of a single execution node. It should be understood that each execution node that processes the same query task in parallel can implement intra-node scheduling according to this concept. Figure 3 is a schematic diagram of the internal logical structure of an execution node provided by an exemplary embodiment of the present disclosure. Referring to Figure 3, in this embodiment, the executor in the execution node can be modified, specifically, the data distribution component (such as the exchange operator) in the executor can be modified, so that the execution node performs intra-node scheduling according to the inventive concept provided in this embodiment, thereby improving the query execution efficiency.

[0041] Referring to FIG3 , when the query object of the query task is the target partition table, in this embodiment, the execution node can determine the target partition to which the subtask is assigned in the target partition table after receiving the subtask. In practical applications, the identifier of the target partition table and the identifier of the target partition assigned to the execution node in the target partition table can be carried in the subtask. In this way, the execution node can determine the target partition assigned in the target partition table by parsing the subtask. Of course, the scheduler in FIG2 can also push the identifier of the target partition to which the subtask is assigned in the target partition table to the execution node. In this embodiment, there is no limitation on how to support the execution node in obtaining the target partition assigned in the target partition table.

[0042] Among them, the target partition table refers to the partition table that the query task needs to query. Similarly, each subtask divided from the query task will also use the target partition table as the query object, but it should be understood that the subtask does not need to query the entire target partition table, but only needs to query the target partition in the target partition table. In addition, the number of target partition tables is related to the task category of the query task. For example, if the subtask is a connection task (also known as a join task), the data in the target partition table is usually two or more; if the query task is an aggregation task (also known as an agg task), the target partition table is usually one. It should be understood that the query task corresponds to the physical plan mentioned above, and its essence can be understood as a plan tree. The task type here is defined based on the root node in the plan tree. For example, if the root node in the plan tree is a join operator, the query task is described as a connection task. For example, if the root node in the plan tree is an agg operator, the subquery task is described as an aggregation task. In this embodiment, other nodes in the plan tree are not limited. For example, if the root node is a join operator, its lower layer is connected to two child nodes, and the two child nodes can be connected to one or more child nodes respectively. Usually, the leaf node of the plan tree is a scan node (scan operator), which is used to pull data for the query task. In this embodiment, the operator type on each child node is not limited. It should also be understood that the task type of the subtask is usually the same as that of the query task. The essence of the subtask is also a physical plan, and usually the planning logic of the subtask is consistent with that of the query task, except that the subtask may only be responsible for querying part of the data required for the query task.

[0043] In this embodiment, the number of target partitions allocated to the current execution node in the target partition table may be one or more, which is determined by the inter-node scheduling process, and this embodiment does not intervene in this.

[0044] In this way, the partition information of the target partition is transparently transmitted to the execution node. Based on this, this embodiment proposes that the execution node use the acquired partition information as the basis for intra-node scheduling, configuring a scheduling relationship between the target partition and each worker process launched for the subtask within the execution node. This scheduling relationship is used to indicate which worker process is responsible for querying which target partition.

[0045] In this embodiment, there is no restriction on the strategy used within the execution node to configure the scheduling relationship based on the partition information. The basic concept here is that within the execution node, intra-node scheduling is performed based on partitions, rather than based on data records as in the traditional shullfe mechanism.

[0046] For example, if the subtask is a connection task, and it instructs to connect the data records with the same id in partition table A and distribution table B, and the six target partitions assigned to the execution node are id=0, id=2, id=4, id=6, id=8 and id=10, then when the execution node performs intra-node scheduling, it can configure the scheduling relationship between these six target partitions and the three work processes started in the execution node for the subtask. For example, each work process can be responsible for two of the target partitions.

[0047] On this basis, the execution node can execute the query according to the execution plan of the subtask. As mentioned earlier, there will be a scan node in the execution plan to pull data from the target partition. Based on the scheduling relationship between the target partition and the worker processes started for the subtask in this execution node, the execution node can schedule the pulled data to the worker process associated with the partition to which it belongs. Continuing with the above example, if there is a scheduling relationship between the target partition with id=0 and worker process 0, then after the execution node pulls the data from the target partition with id=0, it will automatically schedule this data to worker process 0 to execute the query, and no shuffle is required.

[0048] In practical applications, data pulling is typically performed in batches, i.e., pulling data in batches. In this embodiment, the execution node can schedule the batch data to the appropriate work process after completing each batch of data pulling. Of course, the data scheduling operation can also be performed after all the data required for the subtask has been pulled in batches. This embodiment does not limit the timing of the data scheduling operation. Preferably, the data scheduling operation can be performed in batches to improve query execution efficiency.

[0049] In summary, in this embodiment, an improved query solution is proposed under a multi-machine parallel query architecture. After distributing subtasks to multiple parallel execution nodes, each execution node can respectively determine the target partition to which the subtask is assigned in the target partition table. In this way, the partition information in the target partition table can be transparently transmitted to each execution node. On this basis, within the execution node, the scheduling relationship between the assigned partition and each work process started in this node for the assigned subtask can be configured, and the pulled data can be scheduled to the corresponding work process based on the scheduling relationship. That is, within the execution node, the acquired partition information can be fully utilized to perform scheduling within the node. This makes it so that the scheduling process within the execution node no longer needs to rely on the shuffle (redistribution) mechanism, which can effectively save the performance overhead of the scheduling process within the node, thereby improving the query execution efficiency in the multi-machine parallel query scenario.

[0050] Figure 4 is a logical diagram of an implementation scheme for configuring scheduling relationships, provided by an exemplary embodiment of the present disclosure. In the above or following embodiments, various implementations can be employed to support an execution node in configuring the scheduling relationships between target partitions and the various work processes initiated within the execution node for the assigned subtasks. Figure 4 provides a preferred implementation.

[0051] Referring to Figure 4, in this preferred implementation method: the execution node can obtain the identifier of the target partition; map the identifier of the target partition to the partition serial number within this execution node; establish a scheduling relationship between the partition serial number within this execution node and each work process within this execution node; wherein, the partition serial number within this execution node adopts a continuous numbering mechanism.

[0052] The identifier of the target partition is the identifier generated when the partition is defined in the target partition table. For example, "id=1" in the above example can be used as the identifier of a target partition. In this implementation, it is proposed to define the partition serial number within the node for the intra-node scheduling process. It should be understood that for the executor within the execution node, specifically the data distribution component mentioned above, it will recognize the intra-node partition serial number proposed in this embodiment during the intra-node scheduling process, rather than the original identifier of the target partition.

[0053] In this way, the use of a unified partition serial number allows the data distribution component to respond to different subtasks according to common scheduling rules. In addition, the use of a continuous numbering mechanism can avoid problems such as data skew during scheduling within a node, which causes some work processes to be assigned to too many partitions.

[0054] It should be understood that in this preferred implementation, the partitions in the target partition table are not destroyed. Instead, the original partition identifiers in the target partitions are converted into customized partition serial numbers within the execution node, and the execution node will implement intra-node scheduling based on the partition serial numbers.

[0055] Referring to Figure 4, the target partition table contains four partitions: "split0: partition 0", "split1: partition 1", "split2: partition 2", and "split3: partition 4". After the inter-node scheduling process, the two target partitions "split0: partition 0" and "split2: partition 2" are assigned to node0 (execution node 0). Based on this, the aforementioned mapping operation can be performed in node0 to map "split0: partition 0" to partition number 0 and "split2: partition 2" to partition number 1. The same mapping operation is also performed in node1. On this basis, in node0, partition number 0 and partition number 1 can be assigned to the parallel working processes within the node to establish a scheduling relationship between the partition numbers within this execution node and the various working processes within this execution node.

[0056] As shown in the example in Figure 4, in this preferred implementation, the use of a sequential numbering mechanism offers another technical advantage: it can reflect the total number of target partitions assigned to an execution node. This indicator can be used as a parameter for intra-node scheduling, thereby more rationally establishing the scheduling relationship between the partition sequence number within the execution node and the various work processes within the execution node.

[0057] Based on this, further, in the preferred implementation scheme: the execution node can determine the total number of partitions within the execution node based on the partition serial number generated by the mapping operation within the execution node; according to the total number of partitions and the parallelism of the work process within the execution node, the partition serial number that each work process is responsible for is determined to generate a scheduling relationship between the partition serial number within the execution node and each work process within the execution node. Since a continuous numbering mechanism is adopted, after the mapping operation is completed, the difference between the largest partition serial number and the smallest partition serial number is the total number of partitions within the execution node. In actual applications, after determining the total number of partitions, a division calculation can be performed between the total number of partitions and the parallelism of the work process within the execution node to determine how many target partitions each work process can be assigned to, and then the required number of partition serial numbers can be selected for each work process in turn.

[0058] For example, if the partition numbers generated in the execution node are 0, 1, 2, 3, 4 and 5, and the parallelism of the work process started for the subtask in the execution node is 3, then 2 partition numbers can be assigned to each work process. Afterwards, according to the sequential numbering scheme, partition numbers 0 and 1 can be assigned to work process 0, partition numbers 2 and 3 can be assigned to work process 1, and partition numbers 4 and 5 can be assigned to work process 2 to establish a scheduling relationship between the partition numbers in this execution node and the various work processes in this execution node.

[0059] It should be understood that in this preferred implementation scheme, other mechanisms can also be used to establish the scheduling relationship between the partition sequence number within the execution node and the various work processes within the execution node for subtasks. It is not limited to the above-mentioned mechanism based on the total number of partitions and parallelism, and no further examples are given here.

[0060] In addition, in this preferred implementation scheme, if the subtask is a connection task, as mentioned above, the number of target partition tables pointed to by the subtask will be multiple. In this case, in the process of executing the aforementioned mapping operation, the identifiers of the same partitions assigned to this execution node in multiple target partition tables need to be mapped to the same partition serial number within this execution node.

[0061] For example, a subtask executes partition table A and partition table B. Both partition tables are constructed using id as the partition. Partition table A contains partitions with id=0, id=1, id=2, id=3, id=4, and id=5, while partition table B contains partitions with id=0, id=1, id=2, id=3, id=4, and id=5. If the target partition assigned by the execution node is d=0, id=1, and id=2, then the partition with id=0 in partition table A and the partition with id=0 in partition table B must be mapped to the same partition number in the execution node, for example, partition number 0. Similarly, the identifiers of other identical partitions in partition tables A and B should also be mapped according to this principle. This ensures that the partition numbers after mapping maintain the original pairing relationship between partitions even when there are multiple target partition tables.

[0062] It should be understood that the above-mentioned implementation method of supporting the execution node to configure the scheduling relationship between the target partition and the various work processes started for the subtask in the execution node through mapping operations is preferred. This embodiment can also adopt other implementation methods to support the configuration of the scheduling relationship, but is not limited to this. For example, it can be directly allocated according to the identifier of the target partition, etc. No further examples are given here.

[0063] In summary, in this embodiment, various implementation methods can be used to support the direct scheduling relationship between the target partition configured within the execution node and the work processes started for the subtasks within the execution node, thereby realizing intra-node scheduling based on partitions, so that each work process can be assigned to a non-overlapping target partition, thereby executing subtasks in parallel more efficiently and accurately.

[0064] Figure 5 is a logical diagram of an improved query solution provided by an exemplary embodiment of the present disclosure. Referring to Figure 5, after completing the intra-node scheduling, each work process can execute query operations on the partitioned data scheduled thereto according to the execution plan.

[0065] In addition to the aforementioned modifications to the data exchange component (exchange operator) in the execution node, this embodiment also proposes modifications to the hash join operator in the execution node to optimize its performance. In traditional multi-machine parallel query scenarios, the hash join operator in the execution node can only perform hash join operations according to the shuffle mechanism explained above, which results in excessive performance overhead and poor execution efficiency of the hash join operator.

[0066] Referring to the improved query solution provided in FIG5 , in this embodiment, by modifying the hash join operator, the hash join operator can fully utilize the partition information and support independent hash join operations for different partitions. In this way, the hash join operation executed on a single partition no longer needs to involve data in other partitions, avoiding the need for a full table shufle, thereby significantly reducing the performance overhead of the hash join operator and optimizing the efficiency of the hash join operation.

[0067] Based on this, if the query sub-plan is a hash join task, the modified hash join operator can fully utilize the partition information to optimize the efficiency of the hash join operation.

[0068] In this embodiment, a single target partition can be independently managed by a single worker process, and a single target partition can be jointly managed by multiple workers. The following describes the two situations respectively.

[0069] When a single worker process independently manages a single target partition, it can be independently responsible for one or more target partitions. When a single worker process independently manages multiple target partitions, the modified hash join operator can be used to perform separate hash joins on each target partition. Hash joins executed on different target partitions do not interfere with each other.

[0070] For example, if the target process is independently responsible for two target partitions numbered 0 and 1 in partition tables A and B, the target process can use the creation end (i.e., the build end) of the modified hash join operator to construct independent hash tables for these two target partitions in partition table A, respectively, labeling them as hash table 0 and hash table 1. The detection end (i.e., the probe end) of the hash join operator can then be used to detect the target partition numbered 0 in distribution table B based on hash table 0; and to detect the target partition numbered 1 in distribution table B based on hash table 1. Clearly, the modified hash join operator no longer requires full-table detection as required by the traditional shuffle mechanism.

[0071] Continuing with Figure 5, considering that the hash join operator requires partition information, this embodiment further proposes modifying operators in the executor, such as projection operators (e.g., the projet operator) and / or filter operators (e.g., the filter operator), which do not disrupt partitions. This modification primarily preserves the partition sequence numbers of the remaining data as it passes through these operators. This ensures that the remaining data still carries the partition sequence numbers when it is passed to the hash join operator, thereby ensuring that the hash join operator can still perform hash joins based on partitions.

[0072] During their research, the inventors discovered that before the hash join operator, there may be some special operators that can disrupt the partitions, such as the aggregation operator (Agg), the sort operator (sort), and other join operators. Because the data no longer conforms to the original partitioning rules after passing through these special operators, this embodiment discards the associated partition sequence number after the data passes through these special operators, and the hash join operator will fall back to the traditional redistribution shuffle mechanism to perform the hash join operation, ensuring the accuracy of the output results.

[0073] In view of the situation where a single target partition is jointly responsible by multiple workers, this embodiment also proposes a transformation plan for the hash join operator. Here, the target partition shared by multiple work processes is described as a shared partition. During the research process, the inventor found that in the database system, generally speaking, the build process of hash join is a blocking process, that is, the probe must occur after the build is completed, and during the probe process, there will be no update and deletion operations on the hash table, so it is actually a completely read-only process. Based on this, the probe process does not involve the problem of multi-process concurrency control. For this reason, the focus of the transformation in this embodiment is placed on the build process. That is, a transformation plan for the hash join operator is provided to solve the problem of multi-process concurrency control in this case.

[0074] Traditional solutions to multi-process concurrency control problems are usually implemented based on locking mechanisms, etc., and we will not elaborate on these existing solutions here. In this embodiment, a new solution is implemented by modifying the hash join operator:

[0075] Use the hash join operator in the target process to obtain the total number of batches corresponding to the shared partition and the total number of worker processes responsible for the shared partition;

[0076] Use the hash join operator to determine the data batch assigned to the target process itself according to the preset allocation rules, based on the total number of batch processing times and the total number of working processes;

[0077] The hash join operator is used to determine whether the current data batch is the data batch assigned to the target process. If so, the build end of the hash join operator is started to create a hash table for the current data batch to support hash join calculation, so that the various working processes that are jointly responsible for the shared partition can collaboratively create the hash table corresponding to the shared partition.

[0078] In this new solution, a preset allocation rule based on the total number of batches and the total number of worker processes is defined within the hash join operator. This allows each hash join operator, which is jointly responsible for a shared partition, to determine its assigned data batch according to the preset allocation rule after obtaining the total number of batches corresponding to the shared partition and the total number of worker processes responsible for the shared partition. As mentioned earlier, during the execution of subtasks, data is pulled in batches. The total number of batches corresponding to the shared partition can be pre-generated within the scheduler within the execution node or in the data distribution component and notified to each hash join operator jointly responsible for the shared partition. Similarly, the total number of worker processes responsible for the shared partition can be notified to each hash join operator jointly responsible for the shared partition by the data distribution component. This allows each hash join operator jointly responsible for the shared partition to obtain these two parameters and determine its assigned data batch.

[0079] On this basis, from the perspective of a single hash join operator, it can determine whether the current data batch is the data batch to which it should be assigned. If so, it will start the creation of the hash join operator's build end to create a hash table for the current data batch to support hash join calculations. This not only ensures that there is no overlap between the hash tables constructed by the hash join operators that are jointly responsible for the shared partition, but also ensures that no data in the shared partition is missed, thereby effectively solving the problem of multi-process concurrency control.

[0080] This new solution does not limit the specific mechanism of the preset allocation rules. For example, a load balancing mechanism can be used to allocate each hash join operator to a substantially equal number of data batches. Of course, a preemption mechanism can also be used, where each hash join operator preempts the current data batch. The operator that preempts the current data batch is responsible for building the hash table for the current data batch. Further examples of mechanisms are not provided here, and this embodiment is not limited to this. Subsequently, the hash tables created by the hash join operators that are jointly responsible for the shared partition can be merged to generate a hash table corresponding to the shared partition.

[0081] In summary, in this embodiment, a transformation plan for more operators in the execution node in addition to the data distribution component (exchange operator) is proposed, so that after completing the intra-node scheduling using the partition information, the partition information can continue to be fully utilized to optimize the execution performance of multiple operators in the execution plan, thereby improving the execution efficiency of subtasks.

[0082] During their research, the inventors discovered that in traditional multi-machine parallel query architectures, in addition to relying on the shuffle mechanism for intra-node scheduling and hash join operations, data exchange between nodes is also dependent on the shuffle mechanism. Specifically, the execution node must use the shuffle mechanism to determine which data to transfer to other nodes. This creates significant performance overhead within the execution node and reduces query execution efficiency.

[0083] This embodiment further proposes an optimization scheme for the data exchange process between nodes. Data exchange between nodes is typically handled by the data distribution component (e.g., the exchange operator) in the execution node. Therefore, this optimization scheme can be implemented by further modifying the data distribution component (e.g., the exchange operator) in the execution node.

[0084] This optimization solution proposes that the database scheduler, used to distribute subtasks, as shown in Figure 2, construct a partition pruning list for the target partition table and distribute it to the execution nodes where each subtask resides. The partition pruning list contains the partitions in the target partition table whose pruning will not affect the output results. The database scheduler can complete the construction and distribution of the partition pruning list before distributing subtasks to ensure that the execution nodes where each subtask resides can obtain the partition pruning list in a timely manner.

[0085] In actual applications, the database scheduler can identify partitions in the target partitioned table that need to be pruned based on the parsing results corresponding to the query statement of the query task. It can also identify partitions in the target partitioned table that have no data based on the attribute description information corresponding to the target partitioned table, and record the identifiers of these two types of partitions in the partition pruning list. Of course, the partition pruning list can also record a list of other partitions whose pruning will not affect the output results, not limited to the two types shown here, and further examples are not provided here.

[0086] On this basis, in this optimization scheme: the execution node can obtain the partition pruning list constructed for the target partition table; if the execution node needs to exchange data with other nodes, it prunes the target partition assigned to the execution node in the target partition table according to the partition pruning list, and then builds exchange data for other nodes based on the remaining partitions; when receiving a data exchange request, the exchange data built based on the remaining partitions is used as the response data. In other words, the target partition assigned to the execution node that is in the partition pruning list is pruned, and the remaining partitions in the target partition assigned to the execution node can be used to build exchange data. The pruned target partition will no longer require component exchange data.

[0087] As mentioned earlier, if the subtask is a join task, there are typically multiple target partition tables. In this case, the execution node can perform simultaneous pruning on multiple target partition tables when applying the partition pruning list. In other words, if there are multiple target partition tables and any target partitions are listed as pending pruning, the execution node can prune all pending pruning partitions assigned to the execution node from all target partition tables.

[0088] For example, if a subtask targets partitions id=0 and id=1 in partition tables A and B, and the data in partition id=0 in partition table A is empty, partition id=0 is included in the partition pruning list. Based on this, after the execution node pulls partitions id=0 and id=1 from partition tables A and B, it can prune partition id=0 in partition table A. At the same time, even though the data in partition id=0 in partition table B is not empty, the execution node will still prune partition id=0 in partition table B.

[0089] In summary, in this preferred solution, the "logical link for component data exchange" within the execution node remains unchanged. However, before initiating this "logical link for component data exchange," the execution node will prune the target partitions assigned to the execution node from the target partition table using a partition pruning list. The remaining partitions after pruning will serve as the operating scope of the "logical link for component data exchange." This effectively reduces the operating scope of the "logical link for component data exchange" within the execution node, thereby effectively reducing the amount of data transferred during data exchange between nodes, further improving query execution efficiency.

[0090] FIG6 is a flowchart of a query method provided by another exemplary embodiment of the present disclosure. The method is applicable to any one of multiple execution nodes that process query tasks in parallel. The query object of the query task is a target partition table. The method can be implemented by an executor in the execution node. Referring to FIG6, the method may include:

[0091] Step 600: After receiving the subtasks split from the query task, determine the target partition to which the subtasks are assigned in the target partition table;

[0092] Step 601: Configure the scheduling relationship between the target partition and each work process started for the subtask in the execution node;

[0093] Step 602: After the data corresponding to the target partition is pulled, the pulled data is scheduled to the work process associated with the partition to which it belongs based on the scheduling relationship, so that the work process can execute the query.

[0094] In an optional embodiment, configuring the scheduling relationship between the target partition and each work process started for the subtask in the execution node includes:

[0095] Obtaining an identifier of the target partition;

[0096] Mapping the identifier of the target partition to the partition serial number within the current execution node;

[0097] Establish the scheduling relationship between the partition sequence number within this execution node and each work process within this execution node;

[0098] Among them, the partition sequence number within this execution node adopts a continuous numbering mechanism.

[0099] In an optional embodiment, when the subtask is a join query task, there are multiple target partition tables, and mapping the identifier of the target partition to a partition sequence number in the current execution node includes:

[0100] The identifiers of the same partitions allocated to the current execution node in the multiple target partition tables are mapped to the same partition serial number in the current execution node.

[0101] In an optional embodiment, establishing a scheduling relationship between the partition sequence number in the execution node and each work process in the execution node includes:

[0102] Determine the total number of partitions in the execution node based on the partition sequence numbers generated by the mapping operation in the execution node;

[0103] According to the total number of partitions and the parallelism of the work processes in this execution node, the partition serial number that each work process is responsible for is determined respectively, so as to generate a scheduling relationship between the partition serial number in this execution node and each work process in this execution node.

[0104] In an optional embodiment, if the subtask is a hash join task and the target process is independently responsible for multiple target partitions, the method further includes:

[0105] If a projection operator and / or a filter operator exists before the hash join operator in the execution plan of the target process, the partition sequence number of the remaining data is retained after the data scheduled to the target process passes through the projection operator or the filter operator;

[0106] In the hash join operator, hash join operations are performed on different target partitions that are independently responsible for them.

[0107] The target process is any working process in the execution node.

[0108] In an optional embodiment, the method further includes:

[0109] If there is a special operator that can disrupt partitions before the hash join operator in the execution plan of the target process, the relevant partition sequence number is discarded after the data passes through the special operator, so that the hash join operator performs the hash join operation according to the redistribution shuffle mechanism.

[0110] In an optional embodiment, if the target process and other working processes are jointly responsible for a shared partition, the method further includes:

[0111] Use the hash join operator in the target process to obtain the total number of batches corresponding to the shared partition and the total number of worker processes responsible for the shared partition;

[0112] Determine the data batch allocated to the target process itself according to the total number of batch processing times and the total number of working processes using the hash join operator in accordance with a preset allocation rule;

[0113] The hash join operator is used to determine whether the current data batch is the data batch assigned to the target process. If so, the creation build end of the hash join operator is started to create a hash table for the current data batch to support hash join calculation, so that the various working processes that are jointly responsible for the shared partition can collaboratively create the hash table corresponding to the shared partition.

[0114] In an optional embodiment, the preset allocation rule includes allocation according to a load balancing mechanism.

[0115] In an optional embodiment, the method further includes:

[0116] Obtaining a partition pruning list constructed for the target partition table, wherein the partition pruning list includes a list of partitions that will not affect query results after being pruned;

[0117] If the execution node needs to exchange data with other nodes, the target partition allocated to the execution node in the target partition table is pruned according to the partition pruned list, and exchange data is formed for the other nodes based on the remaining partitions;

[0118] When a data exchange request is received, the exchange data constructed based on the remaining partitions is used as response data.

[0119] In an optional embodiment, pruning the target partition in the target partition table assigned to the current execution node according to the partition pruning list includes:

[0120] If there are multiple target partition tables and there are partitions to be pruned in the target partitions that are in the partition pruning list, all the partitions to be pruned in the multiple target partition tables that are assigned to the current execution node will be pruned.

[0121] In an optional embodiment, the partition pruning list is generated by a database scheduler that distributes the subtasks. The database scheduler is used to find the partitions in the target partition table that are designated as needing to be pruned based on the parsing results corresponding to the query statement of the query task, and to find the partitions in the target partition table with empty data through the attribute description information corresponding to the target partition table, so as to construct the partition pruning list.

[0122] It should be noted that some of the processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The operation serial numbers such as 601, 602, etc. are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order.

[0123] Figure 7 is a schematic diagram of the structure of an execution node provided by another exemplary embodiment of the present disclosure. As shown in Figure 7, the execution node is any one of multiple execution nodes that processes a query task in parallel, where the query task's query object is a target partition table. The execution node may include: a memory 70, a processor 71, and a communication component 72.

[0124] The processor 71 is coupled to the memory 70 and the communication component 72 and is configured to execute the computer program in the memory 70 to:

[0125] After receiving the subtasks split from the query task, determining the target partition to which the subtasks are assigned in the target partition table;

[0126] Configure the scheduling relationship between the target partition and each work process started for the subtask in this execution node;

[0127] After the data corresponding to the target partition is pulled, the pulled data is scheduled to the work process associated with the partition to which it belongs based on the scheduling relationship, so that the work process can execute the query.

[0128] In an optional embodiment, when configuring the scheduling relationship between the target partition and each work process started for the subtask in the execution node, the processor 71 may be specifically configured to:

[0129] Obtaining an identifier of the target partition;

[0130] Mapping the identifier of the target partition to the partition serial number within the current execution node;

[0131] Establish the scheduling relationship between the partition sequence number within this execution node and each work process within this execution node;

[0132] Among them, the partition sequence number within this execution node adopts a continuous numbering mechanism.

[0133] In an optional embodiment, when the subtask is a connection query task, there are multiple target partition tables. When mapping the identifier of the target partition to the partition sequence number in the current execution node, the processor 71 may be specifically configured to:

[0134] The identifiers of the same partitions allocated to the current execution node in the multiple target partition tables are mapped to the same partition serial number in the current execution node.

[0135] In an optional embodiment, when establishing the scheduling relationship between the partition sequence number in the execution node and each work process in the execution node, it can be specifically used to:

[0136] Determine the total number of partitions in the execution node based on the partition sequence numbers generated by the mapping operation in the execution node;

[0137] According to the total number of partitions and the parallelism of the work processes in this execution node, the partition serial number that each work process is responsible for is determined respectively, so as to generate a scheduling relationship between the partition serial number in this execution node and each work process in this execution node.

[0138] In an optional embodiment, if the subtask is a hash join task and the target process is independently responsible for multiple target partitions, the processor 71 may further be configured to:

[0139] If a projection operator and / or a filter operator exists before the hash join operator in the execution plan of the target process, the partition sequence number of the remaining data is retained after the data scheduled to the target process passes through the projection operator or the filter operator;

[0140] In the hash join operator, hash join operations are performed on different target partitions that are independently responsible for them.

[0141] The target process is any working process in the execution node.

[0142] In an optional embodiment, the processor 71 may also be configured to:

[0143] If there is a special operator that can disrupt partitions before the hash join operator in the execution plan of the target process, the relevant partition sequence number is discarded after the data passes through the special operator, so that the hash join operator performs the hash join operation according to the redistribution shuffle mechanism.

[0144] In an optional embodiment, if the target process and other working processes are jointly responsible for a shared partition, the processor 71 may further be configured to:

[0145] Use the hash join operator in the target process to obtain the total number of batches corresponding to the shared partition and the total number of worker processes responsible for the shared partition;

[0146] Determine the data batch allocated to the target process itself according to the total number of batch processing times and the total number of working processes using the hash join operator in accordance with a preset allocation rule;

[0147] The hash join operator is used to determine whether the current data batch is the data batch assigned to the target process. If so, the creation build end of the hash join operator is started to create a hash table for the current data batch to support hash join calculation, so that the various working processes that are jointly responsible for the shared partition can collaboratively create the hash table corresponding to the shared partition.

[0148] In an optional embodiment, the preset allocation rule includes allocation according to a load balancing mechanism.

[0149] In an optional embodiment, the processor 71 may also be configured to:

[0150] Obtaining a partition pruning list constructed for the target partition table, wherein the partition pruning list includes a list of partitions that will not affect query results after being pruned;

[0151] If the execution node needs to exchange data with other nodes, the target partition allocated to the execution node in the target partition table is pruned according to the partition pruned list, and exchange data is formed for the other nodes based on the remaining partitions;

[0152] When a data exchange request is received, the exchange data constructed based on the remaining partitions is used as response data.

[0153] In an optional embodiment, when the processor 71 prunes the target partition allocated to the current execution node in the target partition table according to the partition pruning list, it may be specifically configured to:

[0154] If there are multiple target partition tables and there are partitions to be pruned in the target partitions that are in the partition pruning list, all the partitions to be pruned in the multiple target partition tables that are assigned to the current execution node will be pruned.

[0155] In an optional embodiment, the partition pruning list is generated by a database scheduler that distributes the subtasks. The database scheduler is used to find the partitions in the target partition table that are designated as needing to be pruned based on the parsing results corresponding to the query statement of the query task, and to find the partitions in the target partition table with empty data through the attribute description information corresponding to the target partition table, so as to construct the partition pruning list.

[0156] Furthermore, as shown in Figure 7 , the execution node also includes other components such as a power supply component 73. Figure 7 only schematically shows some components, which does not mean that the execution node only includes the components shown in Figure 7 .

[0157] It is worth noting that the technical details of the above-mentioned execution node embodiments can be referred to the relevant description of the execution node in the aforementioned system embodiment. In order to save space, they will not be repeated here, but this should not cause a loss of the protection scope of this disclosure.

[0158] Accordingly, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps performed in the above method embodiment.

[0159] The embodiments of the present disclosure further provide a computer program, which, when executed in a computer, causes the computer to execute the steps performed in the above method embodiments.

[0160] The memory in FIG. 7 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, images, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0161] The communication component in Figure 7 above is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0162] The power supply assembly in Figure 7 provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0163] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0165] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0167] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0169] The foregoing is merely an embodiment of the present disclosure and is not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A data query method, characterized in that: Any execution node among a plurality of execution nodes adapted to parallel process a query task, wherein the query object of the query task is a target partition table, the method comprising: After receiving the subtask split from the query task, determining the target partition to which the subtask is allocated in the target partition table; Configuring a scheduling relationship between the target partition and each work process corresponding to the subtask; After the data corresponding to the target partition is pulled, the pulled data is scheduled to the work process associated with the partition to which it belongs based on the scheduling relationship, so that the work process can execute the query.

2. The method according to claim 1, characterized in that Configuring the scheduling relationship between the target partition and each work process corresponding to the subtask includes: Obtaining an identifier of the target partition; Mapping the identifier of the target partition to the partition serial number in the current execution node; Establish the scheduling relationship between the partition sequence number in this execution node and each work process in this execution node; Among them, the partition sequence number within this execution node adopts a continuous numbering mechanism.

3. The method according to claim 2, characterized in that In the case where the subtask is a connection query task, the number of the target partition tables is multiple, and mapping the identifier of the target partition to the partition sequence number in the current execution node includes: The identifiers of the same partitions in the multiple target partition tables allocated to the current execution node are mapped to the same partition sequence number in the current execution node.

4. The method according to claim 2, characterized in that: Establish the scheduling relationship between the partition sequence number in this execution node and each work process in this execution node, including: Determine the total number of partitions in the execution node based on the partition sequence numbers generated by the mapping operation in the execution node; According to the total number of partitions and the parallelism of the working processes in this execution node, the partition serial number that each working process is responsible for is determined respectively, so as to generate a scheduling relationship between the partition serial number in this execution node and each working process in this execution node.

5. The method according to any one of claims 1 to 4, characterized in that: If the subtask is a hash join task and the target process is independently responsible for multiple target partitions, the method further includes: If there is a projection operator and / or a filter operator before the hash join operator in the execution plan of the target process, the partition sequence number of the remaining data is reserved after the data scheduled to the target process passes through the projection operator or the filter operator; In the hash join operator, hash join operations are performed respectively for different target partitions that are independently responsible for; The target process is any working process in the execution node.

6. The method according to claim 5, characterized in that Also includes: If there is a special operator that can disrupt partitions before the hash join operator in the execution plan of the target process, the relevant partition sequence number is discarded after the data passes through the special operator, so that the hash join operator performs the hash join operation according to the redistribution shuffle mechanism.

7. The method according to claim 5, characterized in that If the target process and other working processes are jointly responsible for a shared partition, the method further includes: Use the hash join operator in the target process to obtain the total number of batch processing times corresponding to the shared partition and the total number of working processes responsible for the shared partition; The hash join operator is used to perform the batch processing according to the preset allocation rules and the total number of batch processing times and the work process. The total number determines the data batches allocated to the target process itself; The hash join operator is used to determine whether the current data batch is the data batch assigned to the target process. If so, the creation build end of the hash join operator is started to create a hash table for the current data batch to support hash join calculation, so that the various working processes that are jointly responsible for the shared partition can collaboratively create the hash table corresponding to the shared partition.

8. The method according to claim 7, characterized in that The preset allocation rule includes allocation according to a load balancing mechanism.

9. The method according to any one of claims 1 to 8, characterized in that: Also includes: Obtaining a partition pruning list constructed for the target partition table, wherein the partition pruning list includes a list of partitions that will not affect query results after being pruned; If the execution node needs to exchange data with other nodes, the subtasks are assigned to the target partitions in the target partition table according to the partition pruning list, and exchange data is formed for the other nodes based on the remaining partitions; When a data exchange request is received, the exchange data constructed based on the remaining partitions is used as response data.

10. The method according to claim 9, characterized in that The target partitions to which the subtasks are allocated in the target partition table are pruned according to the partition pruned list, including: If there are multiple target partition tables and there are partitions to be pruned in the target partitions that are in the partition pruning list, all the partitions to be pruned in the multiple target partition tables that are allocated to the subtask are pruned.

11. The method according to claim 9, characterized in that The partition pruning list is generated by a database scheduler that distributes the subtasks. The database scheduler is used to find the partitions in the target partition table that are designated as needing to be pruned based on the parsing results corresponding to the query statement of the query task, and to find the partitions in the target partition table with empty data through the attribute description information corresponding to the target partition table, so as to construct the partition pruning list.

12. An execution node, characterized in that: Includes memory, processor, and communication components; The memory is used to store one or more computer instructions; The processor is coupled to the memory and the communication component, and is configured to execute the one or more computer instructions to execute the query method according to any one of claims 1 to 11.

13. A database system, characterized in that: It includes multiple execution nodes for processing query tasks in parallel, and the multiple execution nodes respectively receive subtasks split from the query tasks. A single execution node starts multiple parallel work processes for the subtasks it receives. The single execution node is used to execute the query method described in any one of claims 1-11.

14. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the query method according to any one of claims 1 to 11.

15. A computer program, when executed in a computer, causes the computer to execute the query method according to any one of claims 1 to 11.

Images