Data query method, computing device, computer-readable storage medium, and computer program product

Abstract

Provided in the present disclosure are a data query method, a computing device, a computer-readable storage medium and a computer program product. The method is applied to a computing node and comprises: receiving a data query task sent by a master node; on the basis of partition information, acquiring partition query data from a data query partition; when it is determined that the number of partition queries of the partition query data is equal to the target number of queries, determining an initial node sorting result on the basis of the partition query data, and determining partition reference data from the initial node sorting result; performing filtering query on un-queried data in the data query partition on the basis of the partition reference data, and updating the initial node sorting result on the basis of the un-queried data, so as to obtain a node sorting result; sending the node sorting result to the master node, such that the master node determines the target query result on the basis of the node sorting result. By means of parallel processing and filtering, the present disclosure improves the query efficiency.

Description

Data query methods, computing devices, computer-readable storage media, and computer program products Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to data query methods, computing devices, computer-readable storage media, and computer program products. Background Technology

[0002] Top-K queries are a common query operation in databases and data processing systems, designed to select the top K elements from a dataset that meet a certain sorting criterion. Specifically, a Top-K query sorts a dataset (such as a database table) according to a specific standard (e.g., numerical value, timestamp, relevance score, etc.) and returns the top K records after sorting.

[0003] However, in large-scale database systems, due to the large amount of data to be queried, inserting and sorting each piece of data can lead to slow query efficiency. Summary of the Invention

[0004] In view of this, this disclosure provides three data query methods. One or more embodiments of this disclosure also relate to a database system, a computing device, a computer-readable storage medium, and a computer program product, to solve the technical defects of slow query efficiency when performing paginated queries on large-scale database systems in the prior art.

[0005] According to a first aspect of this disclosure, a data query method is provided, applied to a computing node, comprising:

[0006] Receive a data query task sent by the master node, wherein the data query task carries the partition information of the data query partition and the target query quantity of the target query result to be queried;

[0007] Based on the partition information, obtain partition query data from the data query partition;

[0008] If the number of partition queries in the partition query data is equal to the target number of queries, the initial node sorting result is determined based on the partition query data, and the partition reference data is determined from the initial node sorting result.

[0009] The unqueried data in the data query partition is filtered and queried according to the partition reference data, and the initial node sorting result is updated according to the unqueried data to obtain the node sorting result;

[0010] The node sorting results are sent to the master node so that the master node can determine the target query result based on the node sorting results.

[0011] According to a second aspect of this disclosure, a data query method is provided, applied to a master node, comprising:

[0012] Determine the initial data query task and parse the initial data query task to obtain the partition information of multiple data query partitions and the target query quantity of the target query result to be queried.

[0013] Based on the target query quantity and the multiple data query partitions, data query tasks are assigned to multiple computing nodes, wherein the data query task includes partition information of the data query partition and the target query quantity;

[0014] The node sorting results of the plurality of computing nodes are received, and the target query result is determined based on the plurality of node sorting results, wherein the node sorting results are determined by each computing node through the data query method applied to the target computing node as described above.

[0015] According to a third aspect of this disclosure, a data query method is provided, applied to a database system, the database system including a master node and multiple computing nodes, wherein the master node determines an initial data query task and parses the initial data query task to obtain partition information of multiple data query partitions and a target query quantity for the target query result to be queried; based on the target query quantity and the multiple data query partitions, the master node allocates data query tasks to the multiple computing nodes, wherein the data query task includes partition information of the data query partitions and the target query quantity;

[0016] Each computing node receives a data query task sent by the master node and determines the node sorting result according to the data query method applied to the target computing node described above.

[0017] The master node receives the node sorting results of the multiple computing nodes and determines the target query result based on the multiple node sorting results.

[0018] According to a fourth aspect of this disclosure, a database system is provided, the database system including a master node and multiple computing nodes, wherein the master node is configured to determine an initial data query task, and parse the initial data query task to obtain partition information of multiple data query partitions and a target query quantity for the target query results to be queried; and allocate data query tasks to the multiple computing nodes according to the target query quantity and the multiple data query partitions, wherein the data query task includes the partition information of the data query partitions and the target query quantity;

[0019] Each computing node is configured to receive a data query task sent by the master node and determine the node sorting result according to the data query method applied to the target computing node described above.

[0020] The master node is configured to receive the node sorting results of the multiple computing nodes and determine the target query result based on the multiple node sorting results.

[0021] According to a fifth aspect of this disclosure, a computing device is provided, comprising:

[0022] Memory and processor;

[0023] The memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the data query method described above.

[0024] According to a sixth aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the data query method described above.

[0025] According to a seventh aspect of this disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the data query method described above.

[0026] This disclosure provides a data query method in one embodiment. A target computing node receives a data query task assigned by a master node, obtains the target query quantity and partition information of the data query partition carried by the data query task, and then retrieves partition query data from the data query partition corresponding to the partition information. If the partition query quantity of the partition query data equals the target query quantity, partition reference data is determined from the determined initial node sorting results. This partition reference data can be used as a filtering condition to filter unqueried data in the data query partition. The filtered data cannot be included in the final target query result. This filtering method eliminates the need to sort every piece of data in the data query partition. Furthermore, while continuously retrieving partition query data from the unqueried data in the data query partition, the unfiltered unqueried data can be used to update the initial node sorting result, obtaining the node sorting result of the target computing node. By sending the node sorting result to the master node, the master node summarizes the node sorting results to obtain the target query result. This method of using partition reference data to filter unqueried data in the data query partition improves data query efficiency and speeds up the query process. Attached Figure Description

[0027] Figure 1 is a schematic diagram of a data query method applied to a target computing node according to an embodiment of this disclosure;

[0028] Figure 2 is a flowchart of a data query method applied to a target computing node according to an embodiment of the present disclosure;

[0029] Figure 3 is a flowchart of a data query method applied to a master node according to an embodiment of this disclosure;

[0030] Figure 4a is a flowchart of a data query method applied to a database system according to an embodiment of this disclosure;

[0031] Figure 4b is a schematic diagram of a directed acyclic graph provided in an embodiment of this disclosure;

[0032] Figure 4c is a schematic diagram of the cooperation between parallelism and between upstream and downstream operators provided in an embodiment of this disclosure;

[0033] Figure 5 is a flowchart of a data query device applied to a target computing node according to an embodiment of the present disclosure;

[0034] Figure 6 is a flowchart of a data query device applied to a master node according to an embodiment of the present disclosure;

[0035] Figure 7 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0036] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0037] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0038] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0039] Furthermore, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0040] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.

[0041] A min-heap is a special type of complete binary tree data structure where the value of each node is less than or equal to the values ​​of its child nodes. This means that the root node is the smallest element in the entire heap. Min-heaps are commonly used to implement priority queues, supporting efficient insertion and deletion operations with a time complexity of O(log n). When building a min-heap, node positions can be adjusted using a "sinking" operation to maintain the heap property.

[0042] Top-K queries are a common query operation in databases and data processing systems, designed to select the top K elements from a dataset that meet a certain sorting criterion. Specifically, a Top-K query sorts a dataset (such as a database table) according to a specific standard (e.g., numerical value, timestamp, relevance score, etc.) and returns the top K records after sorting. This type of query is well-suited for scenarios such as: leaderboards (e.g., score ranking, sales ranking), recommendation systems (e.g., recommending the top K most relevant items), and real-time queries (e.g., the latest K news items, the most recent K transaction records).

[0043] Top-K query syntax: In SQL (Structured Query Language), Top-K queries are typically implemented using the ORDER BY (sorting) and LIMIT (restriction) keywords. Any query can be appended to the end of the SQL statement with the following clause:

[0044] ORDER BY col1 ASC|DESC,col2 ASC|DESC,...,LIMIT offset,len.

[0045] This is for performing a Top-K query, which means first sorting the data by columns col1, col2, ..., coln (ASC indicates ascending order, DESC indicates descending order), and then outputting len rows of data starting from the offset position in the sorted result.

[0046] Below is a classic Top-K algorithm implementation, suitable for single-threaded, small-scale data scenarios; a Top-K algorithm based on a min-heap:

[0047] A min-heap is a binary heap tree data structure in which the value of each node is no greater than the value of its child nodes. The root node, the top of the heap, is always the minimum value in the heap.

[0048] Implement a simple Top-K algorithm using a heap structure, with the following steps:

[0049] 1. Initialize a min-heap H of size K.

[0050] 2. For dataset A, iterate through each input element A[i] and write it into the heap until K elements are written. At this point, the top element of the heap is H[0].

[0051] 3. Continue iterating through the input dataset, comparing and replacing each element with the top element of the heap:

[0052] If the current element is greater than the top element of the heap, the top element is popped from the heap, and the current element is inserted into the heap. The heap is then automatically adjusted to select a new top element; otherwise, the process continues to traverse to the next element.

[0053] 4. After traversal, the K elements in the min-heap are the Top-K elements in the dataset.

[0054] The key principle of the above algorithm, and the basic idea of ​​"comparison and replacement" in step 3, is:

[0055] Taking the example of finding the largest set of Top-K elements in a set, if we want to find the set of Top-K elements in a set, then for any subset of size >= K, if an element is smaller than the Kth largest element of the subset (the Kth largest element among K elements, i.e., the minimum value among these K elements), then the element must not belong to the final set of Top-K elements.

[0056] In the above simple Top-K algorithm implementation, the heap element set H = {h1, h2, ..., h...} of size K is... k},hk <h k-1 <... th (H)=h k .

[0057] Now, iterate through the elements A_i in dataset A and combine them with the original heap element set to form a new set S = {A_i}. i ,h1,h2,...,h k ) compare and replace, if h k i And combined with the h mentioned earlier k <h k-1 <... k The Kth largest element less than the new set H': H' = {A i ,h1,h2,...,h k-1},h k <K th (H'), at this time h k It will definitely not belong to the final Top-K element set.

[0058] To process the Top-K clause in SQL: ORDER BY col1 ASC|DESC, col2 ASC|DESC, ..., LIMIT offset, len, the basic approach is:

[0059] 1. Solve the Top-K problem, where K = offset + len, and construct a min-heap of size K.

[0060] 2. Construct a comparator to handle all comparison operations within heap sort. If a min-heap data structure is used, for the SQL semantics of ORDER BY...LIMIT offset,len, if the sorting is ascending (ASC), the min-heap comparator needs to use the reverse order (in ascending order, the top element H[0] in the min-heap H is the maximum value in the heap; if the current element being traversed in dataset A is less than the top element, the top element is popped from the heap, and the current element is inserted into the heap. At this time, the heap is automatically adjusted, and a new top element is selected. After traversal, the K elements in the min-heap are the smallest Top-K elements in the dataset); if the sorting is descending (DESC), the ascending order is used directly.

[0061] 3. After obtaining the Top-K results using a min-heap-based algorithm, select len ​​rows of data starting from the offset row as the final SQL result output.

[0062] ​​​This disclosure provides three data query methods. This disclosure also relates to a database system, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0063] Referring to Figure 1, Figure 1 illustrates a scenario diagram of a data query method applied to a database system according to an embodiment of the present disclosure.

[0064] Taking database system 102, which includes master node 1022, compute node 1024, and compute node 1026, as an example, the data query method is explained in detail.

[0065] The master node 1022 can determine the initial data query task based on the user's data query request (e.g., the data query request is a Top-K query request), and parse the initial data query task to obtain the partition information of multiple data query partitions and the target query number (K) of the target query result to be queried.

[0066] For example, multiple data query partitions include data query partition 1 and data query partition 2. Therefore, the master node 1022 can assign data query task 1 to the compute node 1024 so that the compute node 1024 can obtain the node sorting result 1 corresponding to data query partition 1, and assign data query task 2 to the compute node 1026 so that the compute node 1026 can obtain the node sorting result 2 corresponding to data query partition 2.

[0067] Specifically, if computing node 1024 obtains node sorting result 1, computing node 1024 can be identified as the target computing node and computing node 1026 can be identified as other computing nodes; if computing node 1026 obtains node sorting result 2, computing node 1026 can be identified as the target computing node and computing node 1024 can be identified as other computing nodes.

[0068] In practical applications, the process of compute node 1024 obtaining node sorting result 1 and compute node 1026 obtaining node sorting result 2 are parallel, and the query logic within the compute nodes is the same. Of course, if compute node 1024 obtains node sorting result 1 first, compute node 1024 can send node sorting result 1 to master node 1022 first; or if compute node 1026 obtains node sorting result 2 first, compute node 1026 can send node sorting result 2 to master node 1022 first. In the case of cross-node data transmission (such as sending the node sorting result of compute node 1024 or compute node 1026 to master node 1022), data transmission is achieved through network transmission.

[0069] The master node 1022 integrates and summarizes the received node sorting result 1 and node sorting result 2 to determine the final target query result, which includes K data items.

[0070] The data query method for database systems disclosed herein allows the master node to parse the initial data query task and allocate data query tasks to multiple computing nodes based on the partition information of multiple data query partitions obtained from the parsing and the target query quantity of the target query result to be queried. This enables multiple computing nodes to process the allocated data query tasks in parallel, obtain the node sorting results of multiple computing nodes, and then aggregate the node sorting results of multiple computing nodes at the master node to determine the target query result. This avoids transmitting data from multiple data query partitions to the master node via the network and then performing overall aggregation, significantly reducing the amount of data transmitted over the network and network overhead.

[0071] Referring to Figure 2, Figure 2 shows a flowchart of a data query method applied to a target computing node according to an embodiment of the present disclosure, which specifically includes the following steps.

[0072] Step 202: Receive a data query task sent by the master node, wherein the data query task carries the partition information of the data query partition and the target query quantity of the target query result to be queried.

[0073] In this context, the master node is a node in the database system. In the case of a distributed database system, the master node is responsible for coordinating and managing the operations of the entire database cluster, including data allocation, query scheduling, and transaction processing.

[0074] A data query task can be understood as a subtask split from the initial data query task received by the master node; the partition information of the data query partition is used to determine the data partitions involved in this data query task, and the data query partition is the data partition; the target query quantity can be understood as a parameter specified in the user's query request. The target query quantity is used to specify the number of records to be returned from the query results. This parameter is very necessary for scenarios where results are displayed in pages or where partial results are needed instead of all results.

[0075] Specifically, when the master node receives an initial data query task, it can decompose the initial data query task into multiple subtasks (i.e., data query tasks) and assign these subtasks to different computing nodes. These subtasks carry the partition information of the data query partitions for which each computing node obtains data, as well as the target query quantity of the target query results. Each computing node is responsible for processing the corresponding data from different data partitions in parallel to improve query efficiency.

[0076] In practical applications, users can send data query requests to the database system. Upon receiving a data query request, the database system parses the request, determines the initial data query task, and sends the initial data query task to the master node. This allows the master node to break down the initial data query task into multiple data query tasks and send them to the compute nodes for data querying to obtain the query results.

[0077] In practice, the database receives user query requests, i.e., SQL. For example, a user's SQL might look like this:

[0078] SELECT…ORDER BY col1 ASC|DESC…coln ASC|DESC,...,LIMIT offset,len.

[0079] Therefore, the database can interpret the above query request as: finding the first (offset + len) largest value in the order of columns col1, col2, ..., coln, forming a Top-K problem; where K = offset + len; that is, the initial data query task is: first sort according to columns col1, col2, ..., coln, and then output len ​​rows of data starting from the offset position in the sorted result.

[0080] Step 204: Obtain partition query data from the data query partition based on the partition information.

[0081] Partition query data can be understood as query data obtained from data partitions, and the target computing node sorts the obtained partition query data.

[0082] Specifically, the data query process for a computing node is described in detail. The data query process for each computing node is logically consistent and can be implemented by referring to this data query process. At this time, the current computing node can be understood as the target computing node, and the computing nodes other than the target computing node among the multiple computing nodes are identified as other computing nodes.

[0083] The hierarchical structure of a database system consists of databases, tables (a database contains multiple tables), and partitions (a table is split into multiple partitions). Multiple computing nodes deal with different data partitions under the same table in the same database.

[0084] Therefore, based on the partition information of the data query partition, the target computing node can determine the data partition from which it obtains the query data, and thus obtain the partition query data from the corresponding data partition.

[0085] Step 206: If the number of partition queries in the partition query data is equal to the target number of queries, determine the initial node sorting result based on the partition query data, and determine the partition reference data from the initial node sorting result.

[0086] The initial node sorting result can be understood as the result obtained by the compute nodes sorting the partition query data of the obtained partition query count. The specific sorting methods include, but are not limited to, heap sort, insertion sort, etc.

[0087] For example, when using heap sort to sort partition query data, if the query type of the data query task is a descending order query, a min-heap (where the root node is the smallest element in the heap) can be selected to sort the partition query data; if the query type of the data query task is an ascending order query, a max-heap (where the root node is the largest element in the heap) can be selected to sort the partition query data. Specifically, each heap includes a comparator to implement all comparison operations within the heap sort, so that each time a partition query data is obtained, the elements in the heap can be rearranged using the comparator in the heap to place the newly obtained partition query data into the appropriate position in the heap.

[0088] When using insertion sort to sort partition query data, each partition query data can be retrieved and inserted into the ordered sequence by scanning from front to back or from back to front.

[0089] In one or more embodiments of this disclosure, the target computing node includes multiple scanning operators and multiple sorting operators corresponding to the multiple scanning operators; the scanning operators are used to obtain query data from the data query partition, and the sorting operators are used to sort the obtained query data. Specific implementation methods are as follows:

[0090] The step of obtaining partition query data from the data query partition based on the partition information includes:

[0091] Each scanning operator obtains the query data of each data query partition from each data query partition according to the partition information, and sends the query data of each partition to the corresponding sorting operator;

[0092] When the number of partition queries in the partition query data is determined to be equal to the target number of queries, determining the initial node sorting result based on the partition query data and determining the partition reference data from the initial node sorting result includes:

[0093] When it is determined that the number of partition queries in the partition query data is equal to the target query number, each sorting operator performs heap sort on each partition query data to obtain multiple initial heap sort results, and determines the initial node sorting result based on the multiple initial heap sort results;

[0094] The partition reference data is determined from multiple initial heap sort results corresponding to the initial node sorting results.

[0095] The number of partition queries for partition query data can be understood as the number of partition query data received by the sorting operator inside the target computing node.

[0096] Specifically, when a user initiates a Top-K query, m scan operators are created on N computing nodes. Each computing node has m / N scan operators responsible for pulling data from its respective data partition. The scan operators within each computing node are set to output the data to the downstream m / N Top-K operators (Level 1 Top-K, L1 Top-K).

[0097] For example, if there are 2 compute nodes and 4 scan operators, then two scan operators (such as scan operator 1 and scan operator 2) on a compute node can retrieve query data from their respective data partitions in parallel. For example, scan operator 1 retrieves query data from data partition 1, and scan operator 2 retrieves query data from data partition 2.

[0098] In the case where a scan operator corresponds to an L1 Top-K operator (first-level sorting operator), each scan operator sends the obtained query data to the corresponding first-level sorting operator so that the first-level sorting operator can sort the received query data.

[0099] In practical applications, each computing node has m / N L1 Top-K operators that receive data from the upstream scan operators and calculate the Top-K value of their respective query data through an internally maintained min-heap.

[0100] Scan operator 1 will write Chunk data to the L1 TOP-K operator. A Chunk is the basic data unit in the database executor, which typically contains 1024 rows of data. The L1 TOP-K operator checks each row of the Chunk data. Since the number of elements in the min-heap of the L1 TOP-K operator is 0 at the beginning, the data can be directly added to the min-heap maintained by the L1 TOP-K operator, and the total number of elements written to all L1 Top-K operators by this computing node will be recorded and updated.

[0101] All parallelisms share a single variable, Kth, which represents the currently known Kth largest value globally. This can be a single column value, val_col, or a vector {val_col1, val_col2, ..., val_coln} composed of multiple columns; no restriction is placed on this. As operators collaborate among different parallelisms, Kth gradually converges to the true overall Kth largest value, Kth'. The faster the Kth->Kth' transition, the greater the probability of filtering out data, the less data is written to the heap, and the higher the overall performance.

[0102] Specifically, in the initial state, when the L1 Top-K operator is initialized, the global number of elements is 0. At this time, the current Kth largest value cannot be calculated. Inside the L1 Top-K operator, each time a line is written, the total number of elements written to all L1 Top-K operators in that computing node is updated. This update process is atomic and concurrency-safe. When the total number of global elements equals K (the target query data), for all heaps of size K maintained inside all L1 Top-K operators, the heap sort result (i.e., the initial heap sort result) is determined. By integrating multiple heap sort results, the node sort result of size K (i.e., the initial node sort result) is determined. In this disclosure, since the purpose of determining the initial node sort result is to obtain the current Kth value (the minimum value in the node sort result), since each L1 Top-K operator maintains a heap, and the top element of the heap is the minimum value of each heap, the minimum value is obtained by obtaining the top element of the heap maintained by each L1 Top-K operator, min(H i By calculating their minimum value, we can more quickly obtain the current global Kth value (i.e., the partition reference data). At this point, the initialization of the Kth value is complete.

[0103] The parallelism m corresponds to m threads, and each thread maintains a heap H. i So, when the sum of the number of heap elements on m threads equals K, these heaps together form a set of elements of size K. Taking TOP-K query as an example to find the maximum TOP-K element, min(H i If we consider the minimum value of the heap elements (the top element) for each thread, then for a set S of size K formed by all threads, the Kth largest element of S is: K th =min(min(H1),min(H2),...min(H m )).

[0104] Using the previous example, a computing node includes two scan operators, which correspond to two threads, such as thread 1 and thread 2. When K equals 10, the L1 Top-K operator corresponding to thread 1 maintains a heap H6 of size 6 (with 6 heap elements), and the L1 Top-K operator corresponding to thread 2 maintains a heap H4 of size 4 (with 4 heap elements). At this time, Kth is the minimum value of the top element in heaps H6 and H4.

[0105] The data query method disclosed herein, based on the total number of computing nodes N and the total parallelism m, achieves load balancing by allocating tasks when the parallelism is evenly distributed across the computing nodes. This involves the basic principles of parallel computing and distributed systems, and can ensure even task allocation and maximum resource utilization. In other words, it can achieve efficient parallel computing through each scanning operator and the corresponding sorting operator.

[0106] Step 208: Filter and query the unqueried data in the data query partition according to the partition reference data, and update the initial node sorting result according to the unqueried data to obtain the node sorting result.

[0107] In one or more embodiments of this disclosure, when partition reference data is obtained, the partition reference data can be used to filter unqueried data in the data query partition, reducing the amount of data processing and thus accelerating data query efficiency. Specific implementation methods are described below:

[0108] The step of filtering and querying unqueried data in the data query partition based on the partition reference data, and updating the initial node sorting result based on the unqueried data to obtain the node sorting result includes:

[0109] Based on the partition reference data, using preset filtering conditions, the unqueried data in the data query partition is filtered and queried to obtain the current query data. The initial node sorting result is updated based on the current query data, and the partition reference data is updated based on the updated initial node sorting result.

[0110] If it is determined that there is still unqueried data in the data query partition, the step of filtering and querying the unqueried data in the data query partition using the preset filtering conditions to obtain the current query data continues according to the updated partition reference data, until it is determined that there is no unqueried data in the data query partition, and then the node sorting result is obtained.

[0111] Specifically, when the query type of the data query task is a descending order query, the preset filtering condition can be understood as filtering out unqueried data in the data query partition that is smaller than the partition reference data; when the query type of the data query task is an ascending order query, the preset filtering condition can be understood as filtering out unqueried data in the data query partition that is larger than the partition reference data.

[0112] Specifically, given the partition reference data, the input data of the scan operator is filtered using the partition reference data. Taking the query type of the data query task as a descending order query as an example, the TOP-K query is to find the maximum TOP-K element. The partition query data is the minimum value among the global K elements. Therefore, only data in the unqueried data of the data query partition that is larger than the partition query data is likely to be in the final maximum TOP-K result. That is, it is necessary to filter out the unqueried data in the data query partition that is smaller than the partition reference data.

[0113] If the unqueried data has not been filtered, the data is identified as the current query data. An attempt is made to write the current query data into the heap maintained by the sorting operator and update the initial node sorting result of the target computing node to update the partition reference data. That is, each newly written data attempts to compare and update the Kth value in order to increase the Kth value as much as possible and converge faster, thereby filtering more data and avoiding unnecessary heap writes.

[0114] Repeat the above steps to update the initial node sorting result and the partition reference data. The updated initial node sorting result and partition reference data can be used to filter unqueried data more quickly.

[0115] In one or more embodiments of this disclosure, the target computing node may also combine partition reference data from other computing nodes to filter and query unqueried data in the data query partition, thereby utilizing more favorable query filtering conditions to achieve faster queries. Specific implementation methods are described below:

[0116] The step of filtering and querying unqueried data in the data query partition based on the partition reference data, and updating the initial node sorting result based on the unqueried data to obtain the node sorting result includes:

[0117] Based on the partition reference data and the partition reference data of other computing nodes, the unqueried data in the data query partition is filtered and queried, and the initial node sorting result is updated based on the unqueried data to obtain the node sorting result. The other computing nodes are other computing nodes that execute the data query task, and the method for determining the partition reference data of the other computing nodes is the same as that for the target computing node.

[0118] Specifically, the partition reference data of other computing nodes is determined in the same way as the target computing node. Therefore, following the example above, the partition reference data of other computing nodes is also the global Kth largest value of the corresponding computing node. Since the final target query result is obtained by integrating the node sorting results of each computing node, the largest partition reference data among the multiple partition reference data of multiple computing nodes can be used to update the partition reference data of each computing node, because the largest partition reference data among these computing nodes is closest to the global true Kth largest value.

[0119] In practical applications, after the Kth value within each computing node is initialized and continuously updated, each computing node periodically (usually every tens of milliseconds) broadcasts its current Kth value to other computing nodes. Each computing node then calculates the maximum Kth value and updates it to its local Kth value. th =max(K) th1 ,K th2 ,...K thN This step further accelerates the convergence of the Kth value across the entire computing cluster.

[0120] The data query method provided in this disclosure can achieve faster filtering of unqueried data by utilizing the partition reference data of the target computing node and the partition reference data of other computing nodes (actually, it updates the partition reference data of the target computing node by referencing the partition reference data of each computing node). This is because the global Kth value is often larger than the heap top value on a single thread or inside a single L1 Top-K operator, the update frequency is higher, and the convergence speed is faster.

[0121] In one or more embodiments of this disclosure, the step of filtering and querying unqueried data in the data query partition based on the partition reference data and partition reference data of other computing nodes, and updating the initial node sorting result based on the unqueried data to obtain the node sorting result includes:

[0122] Based on the partition reference data and the partition reference data of other computing nodes, the unqueried data in the data query partition is filtered and queried using preset filtering conditions to obtain the current query data. The initial node sorting result is updated based on the current query data, and the partition reference data is updated based on the updated initial node sorting result.

[0123] If it is determined that there is still unqueried data in the data query partition, the step of filtering and querying the unqueried data in the data query partition using the preset filtering conditions to obtain the current query data continues, based on the updated partition reference data and the partition reference data of other computing nodes, until it is determined that there is no unqueried data in the data query partition, and the node sorting result is obtained.

[0124] In practical applications, when the query type of the data query task is a descending order query, if the partition reference data of other computing nodes is greater than the partition reference data of the target computing node, then the partition reference data of the target computing node will be updated to the partition reference data of other computing nodes. In this case, the preset filtering condition can be understood as filtering out the unqueried data in the data query partition that is smaller than the partition reference data.

[0125] When the query type of the data query task is ascending order query, if the partition reference data of other computing nodes is less than the partition reference data of the target computing node, the partition reference data of the target computing node will be updated to the partition reference data of other computing nodes. The preset filtering condition can be understood as filtering out unqueried data in the data query partition that is larger than the partition reference data.

[0126] Specifically, the implementation logic of this part is similar to that of the above embodiments, and will not be repeated here.

[0127] In one or more embodiments of this disclosure, when the current query data attempts to be written into the initial node sorting result, it can only be inserted into the initial node sorting result and further update the partition reference data if the current query data meets the result update conditions. Specific implementation methods are as follows:

[0128] The steps of updating the initial node sorting result based on the current query data and updating the partition reference data based on the updated initial node sorting result include:

[0129] The current query data is compared with the maximum or minimum value in the initial node sorting result. If the comparison result meets the result update condition, the current query data is inserted into the initial node sorting result, and the maximum or minimum value is removed to update the initial node sorting result. The maximum or minimum value is the maximum or minimum value of the initial node sorting result.

[0130] Determine the maximum and minimum values ​​in the updated initial node sorting results, compare the maximum and minimum values ​​in the updated initial node sorting results with the partition reference data, and if the comparison result meets the reference update conditions, determine the maximum and minimum values ​​in the updated initial node sorting results as the partition reference data.

[0131] In the case where a computing node corresponds to one scan operator and one sorting operator, and a heap is maintained, the initial heap sort result is the initial node sort result.

[0132] Wherein, when the query type of the data query task is a descending order query, the maximum and minimum values ​​are the minimum values ​​of the initial node sorting results, and the result update condition is that the current query data is greater than the minimum value of the initial node sorting results; the reference update condition is that the maximum and minimum values ​​in the updated initial node sorting results are greater than the partition reference data.

[0133] When the query type of the data query task is ascending order query, the maximum value is the maximum value of the initial node sorting result, and the result update condition is that the current query data is smaller than the maximum value of the initial node sorting result; the reference update condition is that the maximum value in the updated initial node sorting result is smaller than the partition reference data.

[0134] Specifically, when the initial node sorting result is determined by the heap sort algorithm and the maintained heap, the maximum and minimum values ​​in the initial node sorting result can be understood as the top element of the heap. When the query type of the data query task is a descending query, the top element of the heap is the minimum value in the heap, and when the query type of the data query task is an ascending query, the top element of the heap is the maximum value in the heap.

[0135] Using the previous example, taking a descending order query as an example, the current query data is compared with the top element of the heap. If the current query data is larger than the top element of the heap, the heap is updated. That is, the current query data is inserted into the initial node sorting result, and the top element of the heap is removed. The heap is automatically adjusted to select a new top element.

[0136] When a new top element is selected, it is compared with the partition reference data. If the number of elements in the heap is equal to K and the new top element is larger than the current partition reference data, then the new top element can be determined as the partition reference data. That is, the number of elements in the heap is equal to K, and the selected new top element is the Kth largest element in the heap. If the Kth largest element in the heap is greater than the current Kth value of the computing node (multiple threads on the same computing node share the Kth value of the computing node), then the current Kth value can be replaced so that Kth gradually converges to the true overall Kth largest value Kth'.

[0137] In practical applications, both the initial node sorting result and the initial heap sorting result are determined by the maintained heap. When there are multiple threads on the same computing node, each currently queried data first attempts to insert into the heap maintained by the L1 TOP-K operator corresponding to the data partition. If there is no unqueried data in each data partition, multiple threads on the same computing node obtain the initial heap sorting result of each thread. At this time, the initial heap sorting result is used as the query data to attempt to insert into the heap maintained by the target computing node. The implementation logic is similar to the above, and the query data is also filtered using Kth. Furthermore, since the data in the initial heap sorting result is an ordered sequence {Chunk_ij}, where i represents that the Chunk comes from the i-th thread / operator and j represents the sequence number of the Chunk in the i-th thread / operator, Chunk_ij is obviously ordered internally, arranged in descending order of the sorting sequence. Therefore, when the L2 Top-K (second-level sorting operator corresponding to the compute node) and L3 Top-K (third-level sorting operator corresponding to the master node) encounter a row of data in Chunk_ij that is less than the current Kth value, all subsequent rows of data will also be less than the Kth value and can be discarded directly, thereby improving processing efficiency.

[0138] In one or more embodiments of this disclosure, because the Kth value corresponding to each thread can be affected by the Kth values ​​of other computing nodes—for example, by using the Kth values ​​of other computing nodes to update the Kth value of this computing node—the Kth value corresponding to each thread can be updated at any time. Based on this, the top element of the heap can be cleaned up at any time using the Kth value corresponding to this thread. Specific implementation methods are as follows:

[0139] After determining the partition reference data from the initial node sorting result, the process further includes:

[0140] The extreme values ​​in the initial node sorting result are compared with the partition reference data. If the comparison result meets the removal condition, the extreme values ​​in the initial node sorting result are removed to update the extreme values ​​in the initial node sorting result.

[0141] Specifically, when the query type of the data query task is a descending order query, the maximum value is the minimum value of the initial node sorting result, and the removal condition is the maximum value in the initial node sorting result that is smaller than the partition reference data; when the query type of the data query task is an ascending order query, the maximum value is the maximum value of the initial node sorting result, and the removal condition is the maximum value in the initial node sorting result that is larger than the partition reference data.

[0142] Specifically, the first step in the writing process does not process the Chunk data obtained by the scanning operator itself. Instead, each Top-K operator first cleans up the top element of the min-heap.

[0143] The process continuously checks the top element of the min-heap. If the Kth object at this level has already been initialized, a comparator is used to compare the value of the top element with that of the Kth object. If the top element is less than the current value of the Kth object, the top element is removed, and the min-heap automatically selects a new top element. If the top element is not filtered out by the Kth value at this level, the process attempts to filter it using the Kth object at a higher level. If the top element is removed, the process continues to check for a new top element. Otherwise, the top element cleanup process ends.

[0144] The data query method disclosed herein filters out more data by performing a heap top cleanup process, thereby determining the final heap sort result of the heap maintenance more quickly.

[0145] In one or more embodiments of this disclosure, since the target computing node includes multiple scanning operators and multiple sorting operators corresponding to the multiple scanning operators, the partition reference data is determined through the multiple scanning operators and multiple sorting operators, and each scanning operator obtains data from each data query partition to achieve parallel processing.

[0146] After determining the partition reference data from the plurality of initial heap sort results, the method further includes:

[0147] Each sorting operator sends the partition reference data to its corresponding scan operator;

[0148] The step of filtering and querying unqueried data in the data query partition based on the partition reference data, and updating the initial node sorting result based on the unqueried data to obtain the node sorting result includes:

[0149] Each scanning operator determines preset filtering conditions based on the partition reference data, and uses the preset filtering conditions to filter and query unqueried data in the data query partition based on the partition reference data.

[0150] Each scanning operator sends the unfiltered, unqueried data to the sorting operator corresponding to each scanning operator;

[0151] Each sorting operator updates the initial heap sort result based on the unfiltered, unqueried data, and updates the initial node sort result based on multiple updated initial heap sort results to obtain the node sort result.

[0152] For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0153] In one or more embodiments of this disclosure, the sorting operator of the target computing node includes three layers of operators, namely, the sorting operator includes a heap sort operator, a computing node sorting operator, and a master node sorting operator, wherein the computing node sorting operator corresponds to the target computing node, and the master node sorting operator corresponds to the master node.

[0154] Each sorting operator performs a heap sort on the query data of each partition to obtain multiple initial heap sort results. Based on the multiple initial heap sort results, the initial node sorting result is determined, including:

[0155] Each heap sort operator performs heap sort on the query data of each partition, obtaining multiple initial heap sort results;

[0156] Each heap sort operator sends its initial heap sort result to the computing node sort operator, and the computing node sort operator determines the initial node sort result based on the multiple initial heap sort results.

[0157] After determining the initial node sorting result based on the multiple initial heap sort results, the computation node sorting operator further includes:

[0158] The computation node sorting operator updates the initial node sorting result to obtain the node sorting result;

[0159] The computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator determines the target query result based on the node sorting result.

[0160] Specifically, the execution logic of the heap sort operator, the compute node sort operator, and the master node sort operator is similar. They all obtain the corresponding sorting results through a heap and filter the data by maintaining the Kth value. This will not be elaborated further here.

[0161] In specific implementation, the heap sort operator corresponds to the L1 Top-K operator in the above embodiment, the computing node sorting operator corresponds to the L2 Top-K operator in the above embodiment, and the master node sorting operator corresponds to the L3 Top-K operator in the above embodiment. The scanning operator of each computing node outputs data to the L1 Top-K operator, the initial heap sort result of the L1 Top-K operator is aggregated to the L2 Top-K operator, and then the node sorting result of the L2 Top-K operator is transmitted to the L3 Top-K operator of the master node through the swapping operator, so as to realize the parallel computing of Top-K.

[0162] The data processing method disclosed herein can fully utilize the advantages of parallel computing within the target computing node through the heap sort operator, and optimize the overall Top-K efficiency through multi-core performance. The computing node sorting operator summarizes the initial heap sort results at each degree of parallelism within the target computing node to obtain the node sorting result within the target computing node. This step avoids transmitting all initial heap sort results to the master node via the network for overall summarization, thereby significantly reducing the amount of data transmitted over the network and network overhead.

[0163] In one or more embodiments of this disclosure, the execution logic inside the heap sort operator, the compute node sort operator, and the master node sort operator is similar. In this case, the compute node sort operator obtains data from the initial heap sort result corresponding to the heap sort operator.

[0164] The computation node sorting operator determines the initial node sorting result based on the multiple initial heap sorting results, and obtains the node sorting result, including:

[0165] The computing node sorting operator obtains result query data from the multiple initial heap sort results;

[0166] If the number of result queries in the result query data is equal to the target number of queries, the initial node sorting result is determined based on the result query data;

[0167] The computation node sorting operator updates the initial node sorting result to obtain the node sorting result, including:

[0168] Determine partition reference data from the initial node sorting results;

[0169] The unqueried result data in the multiple initial heap sort results is filtered and queried according to the partition reference data, and the computing node sorting operator updates the initial node sorting result according to the unqueried result data to obtain the node sorting result.

[0170] Specifically, since a computing node can include multiple threads, and each thread corresponds to a scan operator and a heap sort operator, a computing node includes multiple initial heap sort results. The data in each initial heap sort result is used as the result query data and attempted to be inserted into the heap maintained by the computing node's sorting operator to obtain the node sorting result of the computing node (similar to attempting to insert the partition query data of each data query partition into the heap maintained by the heap sort operator to obtain the initial heap sort result).

[0171] The attempt to insert the result query data into the heap maintained by the computing node sorting operator means that, because the computing node sorting operator has its corresponding Kth value, that is, when the number of result queries equals the target number of queries, the initial node sorting result is determined based on the result query data, and the partition reference data (i.e., the Kth value) is determined from the initial node sorting result. Each result query data will be filtered by the partition reference data. Only if the result query data is not filtered out can it be updated into the heap maintained by the computing node sorting operator (of course, in actual applications, when the result query data is inserted into the heap maintained by the computing node sorting operator, the top element of the heap will also filter the result query data), thereby updating the initial node sorting result, until all result query data has been queried and filtered out. (In actual applications, it is not necessary to query every result query data; some result query data can be directly filtered out; in conjunction with the above embodiment, because the data in the initial heap sorting result is an ordered sequence, therefore, when L2 When the Top-K operator and L3Top-K operator encounter a row of data in Chunk_ij that is less than the current Kth value, all subsequent rows of data will also be less than the Kth value and can be discarded directly. The node sorting result of the computing node is obtained through the heap maintained by the computing node.

[0172] Step 210: Send the node sorting result to the master node so that the master node can determine the target query result based on the node sorting result.

[0173] In one or more embodiments of this disclosure, when a master node assigns data query tasks to multiple computing nodes, the master node summarizes the node sorting results of the multiple computing nodes to obtain the target query result. Specific implementation methods are as follows:

[0174] Sending the node sorting result to the master node so that the master node determines the target query result based on the node sorting result includes:

[0175] The node sorting results are sent to the master node so that the master node can determine the target query result based on the node sorting results and the node sorting results sent by the other computing nodes, wherein the method for determining the node sorting results sent by the other computing nodes is the same as that for the target computing node.

[0176] Specifically, the target computing node is any one of multiple computing nodes, and the other computing nodes are any one or more of the multiple computing nodes; that is, the target computing node and other computing nodes can use the data query method provided in this disclosure to process the data query task sent by the master node in parallel, and the master node summarizes the node sorting results of multiple computing nodes to obtain the target query result.

[0177] In one or more embodiments of this disclosure, the computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator determines the target query result based on the node sorting result, including:

[0178] The computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator can obtain node query data from the node sorting result. If it is determined that the number of node queries in the node query data is equal to the target number of queries, an initial query result is determined based on the node query data, and target reference data is determined from the initial query result. The unqueried result data in the node sorting result is filtered and queried based on the partition reference data, and the initial query result is updated based on the unqueried result data to obtain the target query result.

[0179] When the master node summarizes the node sorting results of multiple computing nodes, the master node sorting operator can obtain node query data from the node sorting results and maintain a target reference data (similar to the partition reference data in the above embodiment, used to filter the obtained data) to use the target reference data for data filtering, thereby speeding up the query efficiency of obtaining the target query results. Specifically, the process by which the master node sorting operator obtains the target query results is similar to the process by which the computing node sorting operator obtains the node sorting results in the above embodiment, and will not be repeated here.

[0180] The data query method disclosed herein includes a special structure within the target computing node comprising multiple layers of sorting operators. These multiple layers of sorting operators can be processed in parallel, with each operator responsible for specific computational tasks and data processing, ensuring the efficiency of parallel queries and the accuracy of results. Furthermore, within each sorting operator, data filtering efficiency can be improved by maintaining and utilizing partition reference data. Through data collaboration between computing nodes and master nodes, the data processing load of the master node can be reduced, significantly reducing the amount of data transmitted over the network and network overhead, thereby improving data query efficiency.

[0181] Referring to Figure 3, Figure 3 shows a flowchart of a data query method applied to a master node according to an embodiment of the present disclosure, which specifically includes the following steps.

[0182] Step 302: Determine the initial data query task and parse the initial data query task to obtain the partition information of multiple data query partitions and the target query quantity of the target query result to be queried.

[0183] Step 304: Based on the target query quantity and the multiple data query partitions, allocate data query tasks to multiple computing nodes, wherein the data query task includes partition information of the data query partition and the target query quantity.

[0184] Step 306: Receive the node sorting results of the multiple computing nodes, and determine the target query result based on the multiple node sorting results, wherein the node sorting results are determined by each computing node through the above data query method.

[0185] For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0186] The data query method disclosed herein allows the master node to assign data query tasks to multiple computing nodes, enabling multiple computing nodes to process data query tasks in parallel. Furthermore, by receiving the node sorting results from multiple computing nodes rather than all data within multiple data query partitions, the master node can significantly reduce the amount of data transmitted over the network and network overhead, thereby improving data query efficiency.

[0187] Referring to Figure 4a, Figure 4a shows a flowchart of a data query method applied to a database system according to an embodiment of the present disclosure, which specifically includes the following steps.

[0188] The database system includes a master node and multiple computing nodes. In step 402, the master node determines an initial data query task and parses the initial data query task to obtain partition information of multiple data query partitions and the target query quantity of the target query results to be queried. Based on the target query quantity and the multiple data query partitions, the master node allocates data query tasks to the multiple computing nodes. The data query task includes the partition information of the data query partitions and the target query quantity.

[0189] Step 404: Each computing node receives the data query task sent by the master node and determines the node sorting result according to the above data query method;

[0190] Step 406: The master node receives the node sorting results of the multiple computing nodes and determines the target query result based on the multiple node sorting results.

[0191] Specifically, taking a descending order query as an example, the data query method will be explained in detail.

[0192] The database system receives a user's data query request, parses the request, determines the corresponding initial data query task, and sends it to the master node. For example, the initial data query task is "find the first K = (offset + len) largest values ​​in the order of columns col1, col2, ..., coln". Based on this, the target query quantity (K) and multiple data query partitions for the initial data query task are determined, and the N computing nodes and the total parallelism m are determined according to the initial data query task.

[0193] The distributed database computing cluster has N nodes, one of which is a master node that initiates queries and acts as a coordinator, while the other nodes are computing nodes. The master node also performs the functions of a computing node. When a user initiates a Top-K query, the database instance creates an execution directed acyclic graph (DAG) based on the total number of computing nodes N and the total parallelism m (as shown in Figure 4b, which illustrates a schematic diagram of a directed acyclic graph provided in this embodiment). Using this directed acyclic graph, the acyclicity and dependency of task scheduling and data processing flows can be guaranteed.

[0194] In this process, the scanning operator of each computing node outputs data to the L1 Top-K operator of the data partition (i.e., the heap sort operator in the above embodiment). The results of the L1 Top-K operator are aggregated to the L2 Top-K operator of the computing node (i.e., the computing node sorting operator in the above embodiment). The results of the L2 Top-K operator are then transmitted to the L3 Top-K operator of the master node (i.e., the master node sorting operator in the above embodiment) through the exchange operator, thus achieving parallel computing of Top-K.

[0195] Assuming parallelism is evenly distributed across the computing nodes, m scan operators are created on N computing nodes (m / N per node), each responsible for fetching data from its respective data partition. Each computing node's scan operator outputs data to m / N downstream L1 Top-K operators. Each computing node's m / N Top-K operators receive data from the upstream scan operators and calculate the Top-K value of their input data using an internally maintained min-heap. After calculation, each computing node's m / N Top-K operators immediately output the result to a downstream L2 Top-K operator; each computing node contains one L2 Top-K operator. After receiving the calculation results from all L1 Top-K operators, each computing node's L2 Top-K operator calculates its Top-K data. Then, through downstream switching operators, network transmission is initiated to converge the calculation results to the master node's L3 Top-K operator. The master node has one L3 Top-K operator. The Top-K operator receives the Top-K data aggregated from each computing node and calculates the final Top-K result. In practical applications, the L3 Top-K operator outputs the result to the limit operator. For example, for the statement "select * from table order by xx limit offset,len", K = offset + len, and the Limit operator is responsible for outputting len rows of data starting from offset.

[0196] For a total data volume of n, the time complexity of solving the Top-K problem with a single core is O(nlogK). Within a single computing node, for a total data volume of n, assuming m threads are used, each thread processes an average of n / m rows of data, and m min-heaps each handle the Top-K problem with a size of K, and another heap is responsible for aggregating the Top-K results from these m min-heaps to obtain the final Top-K result, its time complexity analysis is as follows:

[0197] Each thread processes n / m rows of data, maintaining a min-heap of size K to handle the Top-K problem. Maintaining a min-heap of size K for a data stream has a time complexity of O(nlogK) for a single operation. Therefore, the total time complexity for each thread processing n / m rows of data is: Merge m min-heaps of size K into a single final min-heap to obtain the Top-K of the total data. The computational complexity of this part can be analyzed by merging the m heaps (each of size K): O(Klogm); therefore, the total complexity is...

[0198] The speedup of multi-core Top-K compared to single-core can be approximated as: When the data query method disclosed herein is applied to a large-scale distributed database, the values ​​of n and m are sufficiently large compared to K, thus achieving a speedup. In other words, multi-core Top-K can significantly improve data query efficiency compared to single-core Top-K.

[0199] Specifically, the master node divides the initial data query task into multiple data query tasks and distributes them to multiple computing nodes. Each data query task corresponds to a data partition. On each computing node, there are m / N scan operators that retrieve data from the corresponding data partition. Each scan operator corresponds to an L1 sorting operator (L1 TOP-K operator). That is, the data retrieved by the scan operators will be sorted by the L1 sorting operator. The L1 sorting operator uses a min-heap to sort the data. The comparator inside the min-heap is determined according to the data query task. When the query type of the data query task is a descending query, the comparator is ascending.

[0200] Whenever a row of data is written into the heap of the L1 sorting operator, the total number of elements in the compute node is updated. When the total number of elements in the compute node reaches K, the minimum value among the top elements of the multiple min-heaps of the compute node is determined as the local Kth value of the compute node. At this time, the initialization of the Kth value is completed.

[0201] After initializing the Kth value, the local Kth value shared by the L1 sorting operator of this computing node is determined as the runtime filter and passed to all scan operators. At this time, the scan operators will filter the data through the runtime filter, so that when the scan operators read data, they will read data that is larger than the current Kth value. In actual applications, when the Kth value changes dynamically, the filtering conditions in the runtime filter also change with the Kth value. That is, each time the runtime filter performs filtering, it will obtain the latest Kth value as the filtering condition.

[0202] Specifically, the query engine of a distributed database supports a runtime-filter mechanism. This is a dynamically changing expression constructed by upper-level operators and passed to the scan operator as a predicate condition to filter scanned data. In the embodiment described in the specification, if the user's Top-K query statement uses the format "ORDER BY col1,col2,...,coln LIMIT offset,len", then the Kth value maintained by the L1 Top-K operator can be combined to form a predicate condition: (col1,col2,...,coln)>=Kth, where the Kth value is dynamically changing and corresponds to a vector of x column values: Kth={val_col1,val_col2,...,val_coln}. The specific process is as follows:

[0203] When creating the DAG execution structure, the Kth object shared by the L1 Top-K operators is wrapped as a runtime filter: func = {(col1,col2,...,coln)>=Kth} and passed to all Scan operators. At the beginning of execution, the Kth value has not been initialized, the runtime filter is not effective, and the scan operators ignore the runtime filter. When the Kth value is initialized and continuously updated, the runtime filter begins to take effect. The scan operators will be aware of the effectiveness of the runtime filter and the dynamic changes in the Kth value. Each time, the latest Kth value is used to form the predicate condition (col1,col2,...,coln)>=Kth to filter the data in the Scan operators, thereby improving the overall performance.

[0204] After initializing the Kth value, the L1 Top-K operator does not process each newly written Chunk data. Instead, it uses a comparator to compare the top element of the heap with the Kth value of the current level. If the top element is less than the current Kth value, the top element is removed, and a new top element is automatically selected for the min-heap. If the top element is not filtered out by the Kth value of the current level (i.e., the top element is greater than the current Kth value of the current level), and a higher-level Kth value exists, then the higher-level Kth value is used for filtering. This continues until all top elements (whether the original top element or the new top element) cannot be filtered out.

[0205] When each thread (one thread corresponds to one operator, and one operator maintains its own heap) writes new data within this compute node, or when the Kth value of another compute node is greater than the Kth value of this compute node, the local Kth value within this compute node is updated. (A compute node corresponds to multiple threads m / N, and each thread corresponds to one Kth value. However, within the compute node, each thread's Kth value corresponds to the same value and is shared between threads. Therefore, this local Kth value can be considered as the Kth value on a compute node.) By attempting to compare and update the Kth value, the aim is to maximize the Kth value and achieve faster convergence, thereby filtering more data and avoiding unnecessary heap writes. The following situations allow for updating the local Kth value of this compute node:

[0206] 1. When a thread in a compute node writes data, if the data is not filtered by the local Kth value, a higher-level Kth value, or the current top element of the heap, and the total number of elements in the thread's min-heap is equal to K, the data can be written to the thread's min-heap, and a new top element can be obtained. The new top element is then compared with the local Kth value, and the larger value is determined as the local Kth value.

[0207] 2. Use the larger Kth values ​​of other computing nodes (whether at a higher level or at the same level) to update the local Kth value on this computing node. The Kth values ​​of other computing nodes are determined by periodic broadcasting.

[0208] Referring to Figure 4c, Figure 4c illustrates a schematic diagram of collaboration between parallelism and upstream and downstream operators provided in this disclosure; wherein, the black dashed line represents the L1 Top-K operator sharing and updating the Kth value, and also using the Kth value to filter the data input to the scan operator; the black solid line represents the scan operator using a runtime filter to filter the data, while the L1 Top-K operator updates and maintains the Kth value in the predicate condition; the gray dashed line represents the L2 Top-K operator maintaining the Kth value and providing it for the L1 Top-K operator to perform additional filtering. In practical applications, the current Kth-th largest value Kth_parent, which is maintained in real time in the L2 Top-K operator (Kth_parent is shared and updated among all L2 Top-K operators), is monotonically increasing and visible to the L1 Top-K operator. The L1 Top-K operator can check the value of Kth_parent to help filter input elements, and the principle is the same as Kth value filtering. Since the input element sequence order of L2 Top-K and L1 Top-K is different, the convergence speed of Kth_parent and Kth value may also be different. Therefore, there is a certain probability that Kth_parent converges faster, that is, its value is larger, and thus it can exhibit better filtering performance.

[0209] By utilizing the global Kth value as much as possible to filter out unnecessary heap writes, the global Kth value is often larger than the heap top value on a single thread or within a single L1 Top-K operator, resulting in a higher update frequency and faster convergence speed. To ensure that the Kth value update is concurrently safe, a CAS (Compare and Swap, an atomic data exchange operation) mechanism is used for each thread to ensure the atomicity of the update.

[0210] When a thread in a compute node writes data, if the data is not filtered by the local Kth value, a higher-level Kth value (such as the target reference data maintained by the L3 Top-K operator), or the current top element of the heap, and the total number of elements in the thread's min-heap is less than K, then the data is directly written to the min-heap without updating the Kth value.

[0211] Since the Kth value represents the Kth largest value globally, when the total number of elements in a min-heap is equal to K, the root element of the min-heap can be used as the Kth value because it is at least the Kth largest value in the min-heap. Therefore, Kth can only be updated when the size of the min-heap is equal to K.

[0212] For the L1 Top-K operator, once all the input data has been written, the sorting operator at this level is considered complete and obtains a reverse-ordered sequence of elements. This ordered sequence is then divided into chunks and output to the next level's L2 Top-K operator. The processing steps of the L2 Top-K operator are the same as those of the L1 Top-K operator. However, since the data received by the L2 Top-K operator is ordered at this point, if a row of data in the Chunk_ij is less than the current Kth value of the L2 level, all subsequent rows of data will also be less than the Kth value and can be discarded. Similarly, the L3 Top-K operator works the same way, which will not be elaborated here.

[0213] In the L3 Top-K operator, a min-heap of size K is maintained to obtain the final Top-K result.

[0214] For each level of the Top-K operator (L1 / L2 / L3), if all input data has been written, the operator enters the completion state. At this point, the total number of elements in the heap is less than or equal to K, because the heap top cleanup, comparison, and filtering processes with a Kth value help identify and discard elements that cannot exist in the final Top-K result. The heap top removal operation is continuously performed on this heap until it is empty. The collected heap top elements are an ordered sequence of elements sorted in reverse order by the comparator. This ordered sequence is divided into chunks and output to downstream operators, as follows:

[0215] For each L1 Top-K operator within a computing node, once the data in its data partition has been read and computed, it means that the operator has completed the Top-N computation process. After each L1 Top-K operator is completed, an ordered sequence is immediately output to the L2 Top-K operator in chunks. This process involves data transfer between memory and the L2 Top-K operator.

[0216] For each L2 Top-K operator within a computing node, the Top-N computation process is completed from receiving data from the first L1 Top-K operator until the last L1 Top-K operator is received. Then, the node begins to initiate network transmission using the switching operator to transmit data to the L3 Top-K operator.

[0217] For the L3 Top-K operator on the master node, the Top-N calculation process is completed from receiving data from the first L2 Top-K operator until receiving data from the last L2 Top-K operator. The top element of the heap is continuously collected to obtain an ordered sequence of size K. Starting from the offset position, len elements are obtained, which is the final result of the Top-K query order by xxx limit offset,len.

[0218] Experiments have shown that the data query method provided in this disclosure can obtain the target query results more efficiently than existing query methods. For example, for the same data query task, the data query method provided in this disclosure can obtain the target query results in 8.21 seconds, while other existing query methods take 11.63 seconds, 23.07 seconds, and 353.15 seconds, respectively. That is, the data query method provided in this disclosure can greatly improve the efficiency of data query and improve the overall performance.

[0219] The data query method disclosed herein can quickly filter and query data through the collaboration between the master node and multiple computing nodes, as well as the collaboration between multiple operators, thereby improving the filtering performance of the entire data query process. Furthermore, when the data received by the L2 Top-K operator is ordered, the orderliness of the data can be utilized to improve data processing efficiency and overall performance.

[0220] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a data query device applied to a target computing node. Figure 5 shows a schematic diagram of the structure of a data query device applied to a target computing node according to an embodiment of this disclosure. As shown in Figure 5, the device includes:

[0221] The receiving component 502 is configured to receive a data query task sent by the master node, wherein the data query task carries partition information of the data query partition and the target query quantity of the target query result to be queried.

[0222] Component 504 is configured to retrieve partition query data from the data query partition based on the partition information.

[0223] Component 506 is configured to determine an initial node sorting result based on the partition query data and determine partition reference data from the initial node sorting result when the number of partition queries in the partition query data is equal to the target query number.

[0224] Component 508 is configured to filter and query unqueried data in the data query partition based on the partition reference data, and update the initial node sorting result based on the unqueried data to obtain the node sorting result;

[0225] Sending component 510 is configured to send the node sorting result to the master node so that the master node determines the target query result based on the node sorting result.

[0226] Optionally, the obtaining component 508 is further configured to:

[0227] Based on the partition reference data, using preset filtering conditions, the unqueried data in the data query partition is filtered and queried to obtain the current query data. The initial node sorting result is updated based on the current query data, and the partition reference data is updated based on the updated initial node sorting result.

[0228] If it is determined that there is still unqueried data in the data query partition, the step of filtering and querying the unqueried data in the data query partition using the preset filtering conditions to obtain the current query data continues according to the updated partition reference data, until it is determined that there is no unqueried data in the data query partition, and then the node sorting result is obtained.

[0229] Optionally, the obtaining component 508 is further configured to:

[0230] Based on the partition reference data and the partition reference data of other computing nodes, the unqueried data in the data query partition is filtered and queried, and the initial node sorting result is updated based on the unqueried data to obtain the node sorting result. The other computing nodes are other computing nodes that execute the data query task, and the method for determining the partition reference data of the other computing nodes is the same as that for the target computing node.

[0231] Optionally, the obtaining component 508 is further configured to:

[0232] Based on the partition reference data and the partition reference data of other computing nodes, the unqueried data in the data query partition is filtered and queried using preset filtering conditions to obtain the current query data. The initial node sorting result is updated based on the current query data, and the partition reference data is updated based on the updated initial node sorting result.

[0233] If it is determined that there is still unqueried data in the data query partition, the step of filtering and querying the unqueried data in the data query partition using the preset filtering conditions to obtain the current query data continues, based on the updated partition reference data and the partition reference data of other computing nodes, until it is determined that there is no unqueried data in the data query partition, and the node sorting result is obtained.

[0234] Optionally, the transmitting component 510 is further configured to:

[0235] The node sorting results are sent to the master node so that the master node can determine the target query result based on the node sorting results and the node sorting results sent by the other computing nodes, wherein the method for determining the node sorting results sent by the other computing nodes is the same as that for the target computing node.

[0236] Optionally, the obtaining component 508 is further configured to:

[0237] The current query data is compared with the maximum or minimum value in the initial node sorting result. If the comparison result meets the result update condition, the current query data is inserted into the initial node sorting result, and the maximum or minimum value is removed to update the initial node sorting result. The maximum or minimum value is the maximum or minimum value of the initial node sorting result.

[0238] Determine the maximum and minimum values ​​in the updated initial node sorting results, compare the maximum and minimum values ​​in the updated initial node sorting results with the partition reference data, and if the comparison result meets the reference update conditions, determine the maximum and minimum values ​​in the updated initial node sorting results as the partition reference data.

[0239] The device further includes:

[0240] An update component is configured to compare the maximum and minimum values ​​in the initial node sorting result with the partition reference data. If the comparison result meets the removal condition, the maximum and minimum values ​​in the initial node sorting result are removed to update the maximum and minimum values ​​in the initial node sorting result. Specifically, when the query type of the data query task is a descending order query, the maximum and minimum values ​​are the minimum values ​​in the initial node sorting result, and the removal condition is that the maximum and minimum values ​​in the initial node sorting result are smaller than the partition reference data. When the query type of the data query task is an ascending order query, the maximum and minimum values ​​are the maximum values ​​in the initial node sorting result, and the removal condition is that the maximum and minimum values ​​in the initial node sorting result are larger than the partition reference data.

[0241] Optionally, the acquisition component 504 is further configured to:

[0242] Each scanning operator obtains the query data of each data query partition from each data query partition according to the partition information, and sends the query data of each partition to the corresponding sorting operator;

[0243] When it is determined that the number of partition queries in the partition query data is equal to the target query number, each sorting operator performs heap sort on each partition query data to obtain multiple initial heap sort results, and determines the initial node sorting result based on the multiple initial heap sort results;

[0244] The partition reference data is determined from multiple initial heap sort results corresponding to the initial node sorting results.

[0245] The device further includes:

[0246] The data sending component is configured such that each sorting operator sends the partition reference data to its corresponding scan operator.

[0247] Optionally, the obtaining component 508 is further configured to:

[0248] Each scanning operator determines preset filtering conditions based on the partition reference data, and uses the preset filtering conditions to filter and query unqueried data in the data query partition based on the partition reference data.

[0249] Each scanning operator sends the unfiltered, unqueried data to the sorting operator corresponding to each scanning operator;

[0250] Each sorting operator updates the initial heap sort result based on the unfiltered, unqueried data, and updates the initial node sort result based on multiple updated initial heap sort results to obtain the node sort result.

[0251] Optionally, the obtaining component 508 is further configured to:

[0252] Each heap sort operator performs heap sort on the query data of each partition, obtaining multiple initial heap sort results;

[0253] Each heap sort operator sends its initial heap sort result to the computing node sort operator, and the computing node sort operator determines the initial node sort result based on the multiple initial heap sort results.

[0254] The computation node sorting operator updates the initial node sorting result to obtain the node sorting result.

[0255] Optionally, the transmitting component 510 is further configured to:

[0256] The computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator determines the target query result based on the node sorting result.

[0257] Optionally, the obtaining component 508 is further configured to:

[0258] The computing node sorting operator obtains result query data from the multiple initial heap sort results;

[0259] If the number of result queries in the result query data is equal to the target number of queries, the initial node sorting result is determined based on the result query data;

[0260] And determine the partition reference data from the initial node sorting results;

[0261] The unqueried result data in the multiple initial heap sort results is filtered and queried according to the partition reference data, and the computing node sorting operator updates the initial node sorting result according to the unqueried result data to obtain the node sorting result.

[0262] Optionally, the obtaining component 508 is further configured to:

[0263] The computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator obtains node query data from the node sorting result. If it is determined that the number of node queries in the node query data is equal to the target number of queries, an initial query result is determined based on the node query data, and target reference data is determined from the initial query result. The unqueried result data in the node sorting result is filtered and queried based on the target reference data, and the initial query result is updated based on the unqueried result data to obtain the target query result.

[0264] The above is an illustrative scheme of a data query device applied to a target computing node according to this embodiment. The technical solution of this data query device applied to a target computing node belongs to the same concept as the technical solution of the data query method applied to a target computing node described above. For details not described in detail in the technical solution of the data query device applied to a target computing node, please refer to the description of the technical solution of the data query method applied to a target computing node described above.

[0265] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a data query device applied to a master node. Figure 6 shows a schematic diagram of the structure of a data query device applied to a master node provided in an embodiment of this disclosure. As shown in Figure 6, the device includes:

[0266] The parsing component 602 is configured to determine an initial data query task and parse the initial data query task to obtain partition information of multiple data query partitions and the target query number of the target query result to be queried.

[0267] Allocation component 604 is configured to allocate data query tasks to multiple computing nodes based on the target query quantity and the multiple data query partitions, wherein the data query task includes partition information of the data query partitions and the target query quantity;

[0268] The component 606 is configured to receive the node sorting results of the plurality of computing nodes and determine the target query result based on the plurality of node sorting results, wherein the node sorting results are determined by each computing node through the above-described data query method.

[0269] The above is an illustrative scheme of a data query device applied to a master node according to this embodiment. The technical solution of this data query device applied to a master node belongs to the same concept as the technical solution of the data query method applied to a master node described above. For details not described in detail in the technical solution of the data query device applied to a master node, please refer to the description of the technical solution of the data query method applied to a master node described above.

[0270] Figure 7 shows a structural block diagram of a computing device 700 according to an embodiment of the present disclosure. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0271] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0272] In one embodiment of this disclosure, the aforementioned components of the computing device 700, as well as other components not shown in FIG. 7, may also be connected to each other, for example, via a bus. It should be understood that the computing device structural block diagram shown in FIG. 7 is for illustrative purposes and not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0273] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0274] The processor 720 is configured to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described data query method.

[0275] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the data query method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the data query method embodiments.

[0276] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data query method.

[0277] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are basically similar to the data query method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the data query method embodiments.

[0278] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data query method.

[0279] The above is an illustrative scheme of a computer program product according to this embodiment. The technical solution of this computer program product and the technical solution of the data query method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data query method described above.

[0280] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0281] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The content contained in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0282] For the sake of simplicity, the foregoing method embodiments are described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and components involved are not necessarily essential to this disclosure.

[0283] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0284] The preferred embodiments disclosed above are provided to help illustrate this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit this disclosure to the specific implementations described. Obviously, many modifications and variations can be made based on the content of this disclosure. This disclosure selects and specifically describes these embodiments to better explain the principles and practical applications of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited by the claims and their full scope and equivalents. Industrial applicability

[0285] This disclosure provides a data query method in one embodiment. A target computing node receives a data query task assigned by a master node, obtains the target query quantity and partition information of the data query partition carried by the data query task, and then retrieves partition query data from the data query partition corresponding to the partition information. If the partition query quantity of the partition query data equals the target query quantity, partition reference data is determined from the determined initial node sorting results. This partition reference data can be used as a filtering condition to filter unqueried data in the data query partition. The filtered data cannot be included in the final target query result. This filtering method eliminates the need to sort every piece of data in the data query partition. Furthermore, while continuously retrieving partition query data from the unqueried data in the data query partition, the unfiltered unqueried data can be used to update the initial node sorting result, obtaining the node sorting result of the target computing node. By sending the node sorting result to the master node, the master node summarizes the node sorting results to obtain the target query result. This method of using partition reference data to filter unqueried data in the data query partition improves data query efficiency and speeds up the query process.

Claims

1. A data query method applied to a target computing node, comprising: receiving a data query task sent by a master node, wherein the data query task carries partition information of a data query partition and a target query quantity of a target query result to be queried; obtaining partition query data from the data query partition according to the partition information; in a case where a partition query quantity of the partition query data is equal to the target query quantity, determining an initial node ranking result according to the partition query data, and determining partition reference data from the initial node ranking result; filtering and querying unqueried data in the data query partition according to the partition reference data, and updating the initial node ranking result according to the unqueried data to obtain a node ranking result; sending the node ranking result to the master node, so that the master node determines the target query result according to the node ranking result.

2. The data query method of claim 1, wherein, The filtering and querying of the unqueried data in the data query partition according to the partition reference data, and the updating of the initial node ranking result according to the unqueried data to obtain the node ranking result, comprises: filtering and querying the unqueried data in the data query partition according to the partition reference data using a preset filtering condition to obtain current query data, and updating the initial node ranking result according to the current query data and updating the partition reference data according to the updated initial node ranking result; in a case where there is still unqueried data in the data query partition, continuing to perform the filtering and querying of the unqueried data in the data query partition using the preset filtering condition to obtain current query data according to the updated partition reference data, until a case where there is no unqueried data in the data query partition is determined, and the node ranking result is obtained.

3. The data query method of claim 1, wherein, The filtering and querying of the unqueried data in the data query partition according to the partition reference data, and the updating of the initial node ranking result according to the unqueried data to obtain the node ranking result, comprises: filtering and querying the unqueried data in the data query partition according to the partition reference data and partition reference data of other computing nodes, and updating the initial node ranking result according to the unqueried data to obtain the node ranking result, wherein the other computing nodes are other computing nodes performing the data query task, and the determination method of the partition reference data of the other computing nodes is the same as that of the target computing node.

4. The data query method of claim 3, wherein, The sending of the node ranking result to the master node so that the master node determines the target query result according to the node ranking result, comprises: sending the node ranking result to the master node so that the master node determines the target query result according to the node ranking result and node ranking results sent by the other computing nodes, wherein the determination method of the node ranking results sent by the other computing nodes is the same as that of the target computing node.

5. The data query method of claim 3, wherein, The method further comprises: In a case where the query type of the data query task is descending order query, the maximum value is a minimum value of the initial node sorting result, The result update condition is a condition that the current query data is greater than the minimum value of the initial node sorting result; 6. The data query method of claim 2 or 5, wherein, The reference update condition is a condition that the maximum value in the updated initial node sorting result is greater than the partition reference data. The method further comprises: In a case where the query type of the data query task is ascending order query, the maximum value is a maximum value of the initial node sorting result, 7. The data query method of claim 6, wherein, The result update condition is a condition that the current query data is less than the maximum value of the initial node sorting result; The reference update condition is a condition that the maximum value in the updated initial node sorting result is less than the partition reference data. The method further comprises: After the partition reference data is determined from the initial node sorting result, the method further comprises:

8. The data query method of claim 6, wherein, The maximum value in the initial node sorting result is compared with the partition reference data, and in a case where a comparison result meets a removal condition, the maximum value in the initial node sorting result is removed to update the maximum value in the initial node sorting result, ​ ​ ​ 9. The data query method of claim 1, wherein, ​ ​ In a case where the query type of the data query task is a descending query, the extreme value is a minimum value of the initial node sorting result, and the removal condition is a condition that the extreme value in the initial node sorting result is less than the partition reference data, In a case where the query type of the data query task is an ascending query, the extreme value is a maximum value of the initial node sorting result, and the removal condition is a condition that the extreme value in the initial node sorting result is greater than the partition reference data.

10. The data query method of claim 1, wherein, The target computing node includes a plurality of scanning operators and a plurality of sorting operators corresponding to the plurality of scanning operators; The obtaining of the partition query data from the data query partition according to the partition information includes: The scanning operators obtain the partition query data from the data query partitions according to the partition information, and send the partition query data to the corresponding sorting operators; In a case where the number of the partition query data is equal to the target query number, the initial node sorting result is determined according to the partition query data, and the partition reference data is determined from the initial node sorting result, including: In a case where the number of the partition query data is equal to the target query number, the sorting operators perform heap sorting on the partition query data, to obtain a plurality of initial heap sorting results, and determine the initial node sorting result according to the plurality of initial heap sorting results; The partition reference data is determined from the plurality of initial heap sorting results corresponding to the initial node sorting result.

11. The data query method of claim 10, wherein, After the determination of the partition reference data from the plurality of initial heap sorting results corresponding to the initial node sorting result, the method further includes: The sorting operators send the partition reference data to the corresponding scanning operators; The filtering query of the unqueried data in the data query partition according to the partition reference data, and the updating of the initial node sorting result according to the unqueried data to obtain a node sorting result, including: The scanning operators determine a preset filtering condition according to the partition reference data, and perform filtering query on the unqueried data in the data query partition according to the partition reference data and the preset filtering condition; The scanning operators send the unqueried data that is not filtered to the corresponding sorting operators; The sorting operators update the initial heap sorting result according to the unqueried data that is not filtered, update the initial node sorting result according to a plurality of updated initial heap sorting results, and obtain the node sorting result.

12. The data query method of any of claims 10-11, wherein, The sorting operators include heap sorting operators and computing node sorting operators, and the computing node sorting operators correspond to the target computing node; The sorting operators perform heap sorting on the partition query data, to obtain a plurality of initial heap sorting results, and determine the initial node sorting result according to the plurality of initial heap sorting results, including: The heap sorting operators perform heap sorting on the partition query data, to obtain a plurality of initial heap sorting results. The respective initial heap sorting operators send the respective initial heap sorting results to the computing node sorting operator, and the computing node sorting operator determines the initial node sorting result according to the plurality of initial heap sorting results.

13. The data query method of claim 12, wherein, After determining the initial node sorting result according to the plurality of initial heap sorting results, the computing node sorting operator further comprises: The computing node sorting operator updates the initial node sorting result to obtain a node sorting result. The computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator determines the target query result according to the node sorting result.

14. The data query method of claim 13, wherein, The computing node sorting operator determines the initial node sorting result according to the plurality of initial heap sorting results, comprising: The computing node sorting operator obtains result query data from the plurality of initial heap sorting results; In a case where the result query quantity of the result query data is equal to the target query quantity, the initial node sorting result is determined according to the result query data; The computing node sorting operator updates the initial node sorting result to obtain a node sorting result, comprising: The partition reference data is determined from the initial node sorting result; The unqueried result data in the plurality of initial heap sorting results is filtered and queried according to the partition reference data, and the initial node sorting result is updated according to the unqueried result data to obtain a node sorting result.

15. The data query method of claim 13, wherein, The computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator determines the target query result according to the node sorting result, comprising: The computing node sorting operator sends the node sorting result to the master node sorting operator, so that the master node sorting operator obtains node query data from the node sorting result, determines an initial query result according to the node query data in a case where a node query quantity of the node query data is equal to the target query quantity, and determines target reference data from the initial query result, filters and queries unqueried result data in the node sorting result according to the target reference data, and updates the initial query result according to the unqueried result data to obtain a target query result.

16. A data query method applied to a master node, comprising: determining an initial data query task, and parsing the initial data query task to obtain partition information of a plurality of data query partitions and a target query quantity of a target query result to be queried; allocating data query tasks to a plurality of computing nodes according to the target query quantity and the plurality of data query partitions, wherein the data query task comprises partition information of a data query partition and a target query quantity; receiving node sorting results of the plurality of computing nodes, and determining a target query result according to the plurality of node sorting results, wherein the node sorting result is determined by any one of claims 1-15 for the data query method.

17. A data query method applied to a database system, the database system comprising a master node and multiple computing nodes, wherein, The master node determines the initial data query task and parses the initial data query task to obtain the partition information of multiple data query partitions and the target query quantity of the target query result to be queried. Based on the target query quantity and the multiple data query partitions, data query tasks are assigned to multiple computing nodes, wherein the data query task includes partition information of the data query partition and the target query quantity; Each computing node receives a data query task sent by the master node and determines the node sorting result according to any one of the data query methods in claims 1 to 15; The master node receives the node sorting results of the multiple computing nodes and determines the target query result based on the multiple node sorting results.

18. A computing device, comprising: Memory and processor; The memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the data query method according to any one of claims 1 to 16.

19. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the data query method according to any one of claims 1 to 16.

20. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the data query method according to any one of claims 1 to 16.

Images